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FOREWORD 


Contract  FA67WA-1700  required  a  simulation  program  to  determine  essential 
cockpit  display  characteristics  (fault  warning,  mode  progress  and  flight 
director  display  elements)  for  Category  III  all-weather  approach  and  landing 
operations.  Contract  FA67WA-1700  was  a  continuation  of  previous  work 
conducted  under  Contract  FA64WA-5143  which  evaluated  whole  flight  director 
systems.  The  purpose  of  these  studies  was  to  predict  various  elements  of 
pilot  performance  and  preference  in  an  envisioned  Category  III  environment 
so  that  quantitative,  public  data  will  be  available  for  consideration  by 
procedural  and  hardware  decision  makers  long  before  Category  III  is 
operational. 

The  study  being  reported  herein  was  directed  toward  fault-warning  and 
mode -progress  display,  and  was  conducted  between  20  March  1967  and 
26  April  1967  as  the  first  of  a  series  of  four  studies  under  Contract 
FA67WA-1700.  Two  of  the  remaining  three  studies  on  this  contract  have  been 
planned  to  further  examine  fault -warning  displays  and  procedures.  Since 
previous  work  evaluated  whole  flight  director  systems,  one  study  has  been 
planned  to  evaluate  rising  runway  and  expanded  localizer  features  found  in 
modern  flight  director  systems.  All  four  studies  are  being  reported 
separately  in  order  to  facilitate  the  dissemination  of  the  results. 

The  conduct  of  this  study  required  the  collective  talents  of  many  indi¬ 
viduals.  Lt.  Colonel  James  R.  Nelson,  FAA  SRDS,  served  as  Contract  Technical 
Monitor.  Mr.  R.  D.  Monroe  was  the  Bunker -Ramo  Corporation  Program  Manager. 
Mr.  L.  S.  Griffin,  Mr.  D.  G.  Findley,  and  Mr.  J.  L.  Streeper  performed 
simulator  modification  and  maintenance.  Mr.  J.  E.  Brown,  Mr.  W.  H.  Haase, 
Mrs.  G.  Y.  Sager  and  Mrs.  F.  D.  Wuestenberg,  respectively,  conducted  data 
reduction  and  analysis  and  assisted  in  the  preparation  of  this  report. 
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ABSTRACT 


The  reported  study  was  the  first  of  a  series  of  four  studies  to  examine 
the  feasibility  of  display  and  control  concepts  for  commercial  subsonic  jet 
transport  all-weather  (Category  Hi)  approach  and  landing.  The  study  was 
addressed  primarily  to  fault  warning.  Pilot  detection  of  autopilot  and 
display  system  failures  was  examined  with  three  levels  of  fault  warning 
display  information.  Display  failure  detection  and  pilot  decisions  were 
additionally  examined  as  a  function  of  pilot  task  load,  manual  in  one  axis 
or  automatic.  A  total  of  70S  simulated  IIS  approaches  were  flown  by  18 
commercial  airline  pilots  in  a  Boeing  707-720B  research  simulator. 

Pilot /system  performance  and  preference  data  indicated  that  the  full 
annunciator  display  system  tested  was  required  in  order  to  attain  the  best 
display  failure  and  passive  autopilot  control  failure  detection.  The  failure 
warning  utility  of  mode  progress  information  below  200  feet  of  altitude  on 
the  approach  was  found  to  be  inadequate.  The  data  suggested  that:  (l)  mode 
progress  information  be  de -emphasized,  (2)  manual  control  of  just  one  axis 
causes  pilot  fault-detection  performance  to  deterioriate  compared  to  monitor¬ 
ing  full  autopilot  operation,  (3)  second  failures  following  first  failures 
which  put  the  pilot  into  split-axis  control  were  frequently  missed,  and 
(4)  there  is  not  enough  time  from  100  feet  to  landing  to  allow  any  complica¬ 
ted  land-or-go-around  decision  process.  Some  general  characteristics  of 
fault  warning  displays  were  discussed. 
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INTRODUCTION 


If  a  failure  of  an  airborne  sub-system  occurs,  something  or  someone  (l) 
must  detect  the  fact  that  the  failure  has  happened ,  and  (2)  process  the 
information  in  order  to  (3)  take  corrective  action.  Who  or  what  should 
detect  the  failure,  process  the  information  and  take  corrective  action?  Ihe 
pilot  is  responsible  for  the  safety  of  flight,  life  and  property;  yet,  for 
some  potentially  catastrophic  failures  such  as  autopilot  hardovers,  monitor 
systems  can  correct  the  danger  almost  before  the  pilot  can  sense  the  problem. 
Conceptually,  monitor  systems  seem  to  be  an  ideal  solution  to  the  fault  detec¬ 
tion  and  warning  problem.  There  are,  however,  some  problems  with  monitor 
systems  that  must  be  mentioned. 

Monitor  problem.  First,  there  is  the  problem  of  setting  error  criteria 
that  will  detect  real  system  malfunction.  Typically,  thresholds  of  allowable 
error  are  set.  The  problem  with  the  threshold  approach  is  that  what  may  be 
a  real  error  in  one  case,  may  be  normal  operation  in  another  case.  If  the 
thresholds  are  set  for  the  first  case,  a  false  alarm  occurs  in  the  second. 
Stated  another  way,  as  the  error  tolerance  is  decreased  in  order  to  increase 
the  probability  of  detecting  system  malfunction,  the  probability  that  the 
monitor  will  false  alarm  is  also  increased. 

On  the  surface ,  it  seems  conservative  to  assume  that  stringent  procedures 
will,  be  followed,  and  that  all  "alarms"  will  be  treated  as  real  failures. 

The  realities  of  human  behavior,  however,  suggest  very  strongly  that  if  a  sys¬ 
tem  false  alarms  a  great  deal  of  the  time,  the  urgency  of  that  warning  or 
action  will  be  attenuated.  If  the  monitor  system  takes  action  such  as  discon¬ 
necting  an  autopilot  in  order  to  circumvent  human  foibles,  then  a  false  alarm 
places  the  vehicle  in  manual  control.  Category  III-C  manual  control  at  low 
altitudes  may  be  hazardous.  A  false  alarm,  therefore,  could  place  the  vehicle 
in  unnecessary  jeopardy. 

The  second  problem  with  monitors  has  to  do  with  the  inherent  reliability 
of  monitoring  devices.  The  reliability  of  the  monitoring  device  must  be 
better  than  the  system  it  has  to  monitor.  High  reliability  is  suggestive  of 
simplicity;  yet,  to  meet  all  of  the  demands  of  assessing  error  of  the  entire 
system  during  an  entire  approach  and  landing  monitors  may  become  complex  and 
expensive. 

A  dilemma.  The  fault  warning  is  at  present  in  a  state  of  dilenma.  On  one 
hand,  monitor  systems  can  be  fast,  but  they  have  limitations.  On  the  other 
hand,  pilots  are  comparatively  slow  acting,  but  they  have  extensive  memory  and 
can  apply  much  experience  to  the  situation.  The  solution  to  this  dilemma  has 
to  lie  in  the  successful  integration  (interface)  of  fault  warning  equipment 
and  the  human  pilot. 
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Approach  to  the  Problem 


It  Is  assumed  that  the  equipment  should  serve  the  pilot  because  the  pilot 
is  responsible.  Wherever  possible,  the  pilot  should  assess  the  situation  and 
make  decisions.  Monitoring  equipment  should  provide  the  pilot  vith  the  inf  or- 
nation  he  needs  to  make  good  decisions.  Where  it  can  be  shown  that  the  pilot 
cannot  perform  failure  detection  and  sake  decisions  vith  required  accuracy  in 
the  time  allowed,  the  equipment  will  have  to  assume  those  critical  functions. 

Hie  performance  limits  for  Category  III-C  landings  do  not  allow  a  great 
deal  of  margin  for  error.  Human  performance  guesses  are  not  precise  enough 
to  allow  system  or  procedural  design  based  upon  private  and  unmeasured  experi¬ 
ence.  Precise  human  fault  detection  performance  measurement  within  the  con¬ 
text  of  an  aircraft  instrument  approach  and  landing  is  needed  for  the  intel¬ 
ligent  allocation  of  function  between  the  pilot  and  the  monitoring  systems. 

The  approach  to  the  problem  taken  in  this  experiment  was  simply  to  docu¬ 
ment  existing  pilot  capabilities  of  detecting  and  taking  action  on  certain 
kinds  of  failures.  On  the  basis  of  measured  pilot  performance,  system  and 
procedural  criteria  may  be  specified.  Given  human  performance  expectations, 
monitoring  systems  will  have  to  overcome  any  inherent  weakness  of  the  pilot. 
Hardware  requirements  may  then  be  based  upon  what  is  really  needed  rather 
than  what  would  be  elegant. 

The  Study 

This  study  was  designed  as  a  first  inquiry  into  how  much  fault  warning 
information  a  pilot  really  needs  to  detect  certain  kinds  of  autopilot  and 
display  system  failures.  The  fundamental  hypothesis  of  the  approach  was  that 
pilot  performance  should  significantly  improve  as  more  information  is  added 
to  his  fault  warning  display.  That  point  at  which  performance  did  not  sig¬ 
nificantly  improve  would  represent  the  minimum  amount  of  information  that  tha 
pilot  netds  unless  actual  measured  performance  still  indicated  a  potential 
hazard. 

Levels  of  information.  Three  levels  of  fault  warning  information  were 
constructed.  The  first  level  served  as  experimental  control.  All  first 
level  failures  were  tested  vith  no  monitoring  system  other  than  the  28  volt 
flag  logic  on  the  pilot's  flight  director  system  and  radar  altimeter  instru¬ 
ment.  This  level  documented  basic  pilot  fault  detection  capability  with 
minimal  monitor  system  assistance. 

The  second  level  of  information  added  a  monitor  system  that  illuminated 
a  master  caution  light.  The  cautioi  light  served  only  as  an  alerting  cue. 

One  might  argue  that  a  monitor  systom  would  certainly  be  able  to  annunciate 
a  specific  failure  just  as  easily  ai  illuminating  a  caution  light.  Even 
though  this  argument  may  be  valid,  the  pilot's  display  panel  is  premium  "real 
estate."  If  the  pilot  can  quickly  and  easily  detect  a  failure  once  he  ie 
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alerted  to  the  fact  that  one  has  occurred,  then  additional  panel  clutter  will 
have  been  avoided.  Since  minimum  requirements  were  the  subject  of  the  study, 
this  level  of  failure  warning  information  was  examined. 

At  this  master  caution  level  of  information,  a  second  light  could 
differentiate  between  (l)  flag  type  failures  and  (2)  autopilot  or  display 
failures  that  do  not  cause  flags  to  appear.  The  utility  of  such  a  "Flag" 
light  was  also  examined  within  the  master  caution  level  of  information. 

A  third  level  of  monitor  information  was  added  to  the  caution  light.  Ibis 
level  allowed  the  annunciation  of  system  problems  so  that  the  pilot  was  told 
what  was  wrong.  In  this  level  the  failure  detection  was  done  for  the  pilot 
(assuming  that  there  were  no  problems  in  the  presentation  of  information  on 
the  annunciator  panel).  His  job  was  to  correct  the  situation. 

Monitor  operation.  Control  and  display  system  failures  were  introduced  to 
exercise  the  three  levels  of  fault  warning  information.  When  the  monitor  was 
operating  during  the  second  and  third  levels  of  information,  a  perfect  moni¬ 
tor  was  assumed.  The  monitor  was  assumed  to  take  5  seconds  to  integrate 
enough  error  during  control  system  failures  to  trigger  the  caution  and/or 
annunciator  lights.  Display  system  failures  were  immediately  detected  by 
the  monitor. 

Pilot  task  load.  Previous  work  (Ref.  4)  indicated  that  failures  that  were 
introduced  when  the  pilot  was  in  manual  control  were  hazardous  because  the 
failures  were  not  always  detected.  The  amount  of  failure  annunciation  infor¬ 
mation  needed  by  the  pilot  certainly  could  be  a  function  of  his  task  load. 

In  this  study,  pilot  task  load  was  varied  by  asking  the  pilots  to  continue  in 
split-axis  control  (manual  in  one  axis,  automatic  in  the  remaining  axes) 
after  experiencing  a  control  failure.  Pilot  performance  during  display 
failures  was  then  measured  as  a  function  of  both  the  amount  of  failure  annun¬ 
ciation  information  available  and  the  pilot  task  load. 

Rollout  performance.  In  previous  studies  (Ref.  4)  the  approach  profile 
terminated  at  touchdown.  For  the  present  study  the  simulator  was  modified  to 
include  a  simulation  of  rollout  to  60  knots  in  order  to  gather  baseline  roll¬ 
out  performance  data.  Pilots  were  asked  to  rollout  using  only  the  horizontal 
situation  indicator  raw  heading  and  localizer  information  to  provide  these 
preliminary  data  for  investigating  possible  display  requirements  for  rollout. 

Fault  warning  information  in  mode  progress  displays.  Mode  progress  dis¬ 
plays  are  specified  for  Category  III  operations  so  that  the  pilot  will  be 
informed  of  the  status  of  his  automatic  equipment.  It  is  assumed  that  the 
pilot  will  watch  mode  progress  and  will  notice  when  mode  sequencing  does 
not  take  place  at  the  proper  time.  Mode  progress  displays  may  have  some 
fault  warning  utility  at  the  outer  marker  because  there  is  plenty  of  time  to 


take  remedial  action.  But,  is  there  enough  time  at  50  feet  for  the  pilot  to 
notice  that  the  flare  mode  has  or  has  not  engaged?  If  he  does  notice  non¬ 
engagement,  does  he  have  enough  time  to  do  anything  about  it?  Some  systems 
under  development  seem  to  assume  that  there  is  enough  time  for  the  pilot  to 
take  action  because  they  place  mode  progress  displays  in  prime "real  estate" 
right  next  to  the  ADI. 

The  need  for  mode  progress  displays  is  not  being  questioned.  What  is  being 
questioned  is  the  intrinsic  failure  alerting  properties  of  mode  progress 
information.  If  mode  progress  displays  can  be  used  only  when  there  is  plenty 
of  time,  should  these  displays  be  placed  in  an  area  close  to  the  ADI  where 
the  highest  priority  information  is  needed?  To  test  the  intrinsic  fault  warn¬ 
ing  value  of  mode  progress  displays,  the  autopilot  flare  mode  engagement  was 
failed  at  50  feet.  The  only  indication  of  mode  non-engagement  was  the  non- 
illumination  of  the  flare  mode  annunciator.  No  other  fault  warning  information 
was  present  during  these  failures. 

Effect  of  shearwinds  upon  touchdown.  Finally,  the  pilots  were  requested  to 
continue  to  touchdown  unless  a  second  failure  disabled  their  display  in  an 
axis  in  which  an  autopilot  failure  had  occurred,  thus  requiring  them  to  fly 
that  axis  manually.  The  experiment  was  designed  so  that  touchdowns  occurred 
during  three  shearwind  conditions:  (l)  tailwind  shear,  (2)  lateral  shear,  and 
(3)  a  combination  of  tailwind  and  lateral  shear.  The  effect  of  these  winds 
upon  touchdown  was  documented. 

When  examining  the  data  of  this  experiment,  the  reader  should  remember  one 
important  point:  Each  pilot  flew  thirty -nine  approaches  for  record.  Very 
early  in  the  experiment  the  subject  pilots  must  have  realized  that  failures 
were  going  to  occur  on  every  approach.  Even  though  the  pilots  were  constantly 
looking  for  failures ,  performance  differences  with  the  various  monitor  modes 
were  marked. 
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METHOD 


Summary 

Three  levels  of  failure  warning  were  compared  (see  Table  M-l).  The  first 
level  was  nothing  more  than  normal  instrument  flags.  The  second  level  assumed 
a  monitor  that  turned-on  a  master  caution  light.  The  third  level  was  an 
elaborate  system  composed  of  instrument  flags,  master  caution  light,  and  an 
annunciator  paneL 


TABLE  M-l 
STUDY  VARIABLES 


•  Amount  of  failure  warning  information 

•  No  monitor,  instrument  flags  only 

•  Master  caution  +  flags 

•  Annunciator  +  master  caution  +  flags 

•  Type  of  failure  introduced  between  450  and  50 
feet  of  altitude 

•  Autopilot 

•  Passive  (fail  dead) 

•  Softover  .25  degrees/second 

•  Flare  mode  engagement 

•  Display  failures  during  automatic  and  split 
axes  control 

•  ADI  flags 

•  HSI  flags 

•  Radar  Altimeter  flags 

•  Vertical  gyro  unreliable  failure 

J.8  pilots  flew  39  approaches  for  record,  total¬ 
ing  702  recorded  approaches  which  started  12  n.m. 
out  on  localizer  course. 
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Two  failure!  per  approach  were  given  during  432  of  the  702  approaches  la 
this  study.  Study  II,  conducted  in  the  STIR  facility,  revealed  that  pilots 
frequently  missed  the  second  failure  when  the  first  failure  put  then  Into 
split-axis  control.  The  second  failure  was  Inserted  no  sooner  than  20 
seconds  after  the  first.  When  the  second  failure  occurred,  the  pilot  was 
required  to  make  aland  or  go- around  decision."  If  the  display  failure  effec¬ 
ted  the  information  he  was  using  to  manually  fly  the  aircraft,  he  was 
required  to  go-around. 

lhe  level  of  monitor  system  required  was  evaluated  toy  counting  the  number 
of  failures  that  the  pilots  did  not  detect  (no  detection),  the  number  of 
Incorrect  responses,  and  the  number  of  Incorrect  decisions  to  land  (manually 
flying  failed  information).  Other  measures  were  longitudinal  range  used, 
lateral  displacement,  pitch  and  roll  attitude,  heading  error,  lateral  drift, 
indicated  airspeed,  vertical  velocity  and  pilot  response  time. 

The  remaining  portions  of  the  method  section  describe  in  greater  detail 
how  the  study  was  conducted. 

Subject  Pilots 

Four  subject  pilots  participated  in  preliminary  flying  for  the  purpose  of 
de-bugglng  the  experimental  procedures,  the  measurement  system,  the  simula¬ 
tion  and  the  questionnaire ;  their  assistance  was  invaluable.  Eighteen  subject 
pilots  flew  for  record.  The  average  pilot  was  a  commercial  Captain  (or 
equivalent),  47  years  old,  with  16,794  hours  of  total  flight  time,  10,345 
hours  of  transport  aircraft  time,  2,922  hours  of  jet  time,  1,494  hours  of 
instrument  time,  and  194  hours  of  simulator  time.  The  pilots  represented 
American  Airlines,  Continental  Airlines,  Eastern  Airlines,  Federal  Aviation 
Administration,  Flying  Tiger  Line,  Fan  American  Airways,  Trans  World  Airlines, 
United  Air  Lines,  and  the  United  States  Air  Force. 

Apparatus 

The  study  was  conducted  using  the  FAA/B-RC  Simulator  for  Transport  Instru¬ 
mentation  Research  (STIR).  STIR  is  a  fixed  base,  aerodynamic  six-degree-of- 
freedom  device  that  simulates  a  Boeing  707-720B  aircraft.  In  order  to 
achieve  the  necessary  performance  accuracy,  the  simulator  is  limited  to  the 
low-speed  (170  knts.),  low-altitude  (2,500  ft.),  small-angle  pertubation 
(±10  deg.)  approach-and-landing  flight  regime.  An  US  final  approach  course 
similar  to  Los  Angeles  International  Airport  including  glide slope,  locali¬ 
zer,  and  marker  beacons  is  simulated. 

The  simulator  system  can  be  functionally  diagrammed  into  four  functional 
groups  shown  in  Figure  M-l.  The  full  system  capability  within  each  of  the 
four  basic  groups  (analog  computation,  program  control  console,  measurement 
equipment,  and  control  cabin)  has  been  documented  elsewhere  (Ref.  4).  For 
the  purpose  of  understanding  the  present  study,  it  is  important  to 
specify  the  control  cabin  configuration. 
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Control  Cabin.  The  control  cabin  exterior  is  a  section  of  a  production 
707-720B  aircraft  with  frosted  windows  to  eliminate  visual  cues.  The 
Captain  and  First  Officer  crew  stations  are  mechanized  t&h  active  controls 
and  instruments  as  shown  in  Figure  M-2. 

Controls.  All  of  the  normal  aerodynamic  surface  controls  are  simulated 
(operational),  and  the  "feel"  is  identical  to  the  normal  707-720B  aircraft 
within  the  performance  limitations.  Other  operational  controls  include 
the  toe  brakes  for  differential  braking,  landing  gear,  spoilers  and  engine 
control  including  thrust  reversal.  Special  controls  shown  in  Figure  M-2 
are  (l)  a  split-axes  autopilot  selector,  (2)  a  pushbutton  flight  director/ 
autopilot  mode  selector,  (3)  an  autothrottle  speed  selector/engage  panel, 
and  (4)  several  special-purpose  switches.  The  autopilot  is  a  simulated 
Bendix  FB-20D  PALS  with  added  split-axes  capability.  The  autopilot  derives 
its  information  from  sensors  which  feed  into  the  First  Officer's  displays 

Displays .  Standard  flight  instruments  and  experimental  fault  warning/mode 
progress  displays  are  mounted  on  the  instrument  panel  as  shown  in  Figure  M-2. 
The  standard  instruments  operate  normally.  The  input  signals  and  flags  can 
be  affected  by  failure  insertions.  The  Captain's  flight  director  system  is 
a  Lear-Siegler  4058G  (Figure  M-3)  which  was  independent  of  the  First  Officer 'b 
flight  director  system.  Signal  flow  diagrams  appear  in  Figures  M-4  and  M-5. 
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FIGURE  M-4  -  Logic  Flew  for  Captain's 


inputs  flight  computers  results 
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The  fault  warning/mode  progress  displays  consisted  of  three  panels.  In 
addition,  a  series  of  four  green  lights  indicated  the  engaged  automatic  con¬ 
trol  equipment.  The  operation  of  the  individual  indicators  follows: 


r 

FLAG 

CAUTION 

DEVIATION 

CAPT 

INI  T 

GO-AROUND 

G/S 

SPD 

m 

G/S 

MDA 

— 

FLR 

ROLLOUT 

FIGURE  M-6  -  Master  Fault  Warning  /Mode  Progress  Display. 


(l)  The  master  fault  warning/mode  progress  display  (Figure  M-6)  was  loca 
ted  above  the  Captain's  flight  director.  The  panel  face  was  opaque 
so  that  the  pilot  could  only  see  the  illuminated  elements  which  are 
listed  below: 

♦FLAG  -  A  master  warning  light  used  to  indicate  that  an  instrument 
flag  was  in  view. 

♦CAUTION  -  A  master  warning  light  used  to  indicate  the  presence  of  a 
failure. 

♦DEVIATION  -  A  master  warning  light  used  with  LOC,  G/S,  SPD  lights. 

♦LOC.  Indicated  excessive  deviation  (  t-k  dot) 

♦G/S.  Indicated  excessive  deviation  (  dot) 

♦SPD.  Indicated  excessive  deviation  (  -5  knts.) 

CAFT.  and  IN IT.  -  Indicated  localizer  engaged. 

TRK  and  G/S  -  Indicated  G/S  engaged  . 

MDA  -  Indicated  100  ft.  altitude  (wheel  height). 

FLR  -  Indicated  flare  mode  engagement. 

ROLLOUT  -  Indicated  mode  engagement. 


(♦) Denotes  that  the  indicator  usage  was  a  study  variable. 
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The  automatic  control  annunciator  panel  (Figure  M-7)  consisted  of  four 
lighted  pushbutton  switches.  If  a  pitch  or  roll  control  failure 
occurred,  the  top  portion  of  the  switch  would  light  up  (red).  If  that 
axis  was  disconnected,  the  bottom  portion  would  light  (amber)  and  the 
top  portion  would  extinguish.  The  pushbutton  was  used  to  disconnect 
the  fault  axis.  The  autothrottle  and  yaw  units  were  not  used. 


CCNI 


FIGURE  M-7  -  Automatic  Control  Annunciator  Panel 


(3)  The  system  annunciator  panel  (Figure  M-8)  unit  indications  (amber)  were 
used  if  either  the  unit  or  one  of  its  primary  inputs  failed.  The  logic 
sequence  is  also  presented  in  the  system  diagrams  (Figure  M-3  and  M-4). 


'  R/A  1 

R/A  2 

FLARE  1 

FLARE  2 

VG  1 

1  VG  2 

DG  1 

DG  2 

F/D  PITCH  1 

F/D  PITCH  2 

F/D  ROLL  1 

F/D  ROLL  2 

FIGURE  M-8  -  System  Annunciator  Panel 


Flight  Profile 


The  flight  profile  was  a  simulated  ILS  approach  to  runway  25L  at  Los 
Angeles  International  Airport.  The  2.75-degree  glideslope  projected  from 
a  point  1,000  feet  inside  the  runway  threshold;  the  -0.7-degree  beam  width 
indicated  -2  dots  of  glide sikpe  deviation  on  the  ADI.  The  t2-degree  locali¬ 
zer  terminated  1,000  feet  beyond  the  departure  end  of  the  runway;  150- 
microamp  beam  width  was  equal  to  two  dots  of  localizer  error  on  the  HSI. 

Initial  conditions.  Prior  to  the  start  of  the  problem  the  simulator  was 
set  into  a  trim  condition  without  autopilot  coupling  approximately  12.5  miles 
from  the  glideslope  shack  at  an  absolute  altitude  of  2,500  feet  (2,628  feet 
barometric  altitude)  and  inbound  on  the  localizer  course.  In  this  position, 
the  simulator  was  two  dots  below  the  glideslope,  the  heading  was  2 48  degrees, 
thrust  was  set  for  147  KIAS  and  flaps  were  set  to  30  degrees.  As  an  initial 
wind  was  inserted,  the  aircraft  heading  was  altered  so  that  lateral  drift 
was  zero.  The  steady-state  wind  was  10  knots  lateral,  10  knots  longitudinal, 
or  a  combination  thereof  in  the  horizontal  plane.  Wind  gusts  were  random 
movements  with  a  maximum  amplitude  of  2  knots  and  a  one-CPS  band  width. 

Profile  execution.  When  the  approach  was  started,  the  autopilot  was 
engaged  in  the  altitude  hold  and  localizer  track  modes.  Airspeed  was  reduced 
to  132  KIAS  and  the  autothrottle  system  was  engaged.  The  autopilot  auto¬ 
matically  captured  the  glideslope  and  trimmed  the  aircraft  for  the  approach. 
At  50  feet  of  absolute  altitude,  the  flare  mode  was  engaged.  At  20  feet,  the 
autothrottle  system  began  a  programmed  linear  power  reduction  to  engine  idle 
at  touchdown.  A  flare  computer  controlled  pitch.  Touchdown  necessitated 
manual  (l)  disengagement  of  the  autopilot  and  the  autothrottle  system, 

(2)  deployment  of  spoilers,  (3)  application  of  reverse  thrust  and  braking, 
and  (4)  use  of  differential  brakes  and/or  rudder  control  until  60  KIAS  reset 
the  simulator  to  the  initial  conditions. 

The  execution  of  a  missed-approach  was  also  a  manual  function.  When  go- 
around  mode  was  selected,  the  flight  director  commanded  eight  degrees  of 
pitch  attitude  and  wings  level.  The  simulator  reset  to  the  initial  conditLcns 
when  the  aircraft  passed  500  feet  of  altitude. 


System  Failures 

In  order  to  exercise  the  fault  warning  system,  autopilot  control  failures, 
Captain's  display  failures  and  flare  mode  engagement  failures  were  inserted 
during  the  approaches.  The  procedures  sub-section  explains  the  experimental 
rules  for  failure  insertion.  A  description  of  each  failure  follows: 
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Control  faLures.  Three  types  of  control  system  failures  were  investiga¬ 
ted: 

(1)  Passive  (dead)  control  failure.  During  passive  autopilot  failures, 
the  autopilot  failed  to  respond  to  its  input  signals  in  either  pitch  or 
roll. 

(2)  Soft over  control  failures.  During  soft  over  autopilot  failures,  the 
autopilot  failed  to  respond  to  its  input  signals  in  either  pitch  or  roll, 
and  produced  aileron  or  elevator  control  movement.  In  pitch,  the  column 
moved  fore  or  aft  0.5  inches/second,  which  was  approximately  equal  to  1.0 
degree  of  elevator;  1.0  constant  elevator  was  equivalent  to  0.25  degrees/ 
second  of  pitch  attitude  rate.  In  roll,  the  wheel  moved  right  or  left  1.0 
degree/second.  One  degree  of  constant  wheel  change  was  approximately  equal 
to  0.3  degrees/second  roll  attitude  rate. 

(3)  Flare  mode  engagement  failure.  At  50  feet  of  absolute  altitude,  the 
flare  mode  failed  to  engage.  During  this  failure,  the  autopilot  continued 
to  track  in  the  glideslqpe  mode.  The  simulator  failed  to  flare,  pitch  atti¬ 
tude  decreased  very  slightly  as  ground  effect  increased,  the  autothrottle 
system  failed  to  reduce  thrust  at  20  feet  of  absolute  altitude,  and  the  flare 
mode  engagement  light  failed  to  illuminate. 

Display  failures .  Six  Captain's  display  failures  were  inserted. 

(1)  Glideslope  receiver  output  opened,  causing  a  glideslqpe  flag  to 
appear  and  the  raw  glideslope  indicator  and  the  pitch  command  bars  to  drive 
to  their  center  positions. 

(2)  Localizer  receiver  output  opened,  causing  the  raw  localizer  course 
deviation  indicator  and  the  expanded  localizer  indicator  to  return  to  their 
center  positions.  The  bank  steering  bar  biased  out  of  view.  Localizer  and 
roll  flight  director  flags  appeared. 

(3)  Directional  gyro  failure  caused  the  horizontal  situation  indicator 
(HSI)  compass  card  to  freeze  and  the  OFF  flag  to  appear. 

(4)  Vertical  gyro  dead  failures  caused  the  vertical  gyro  to  freeze  in 
position  in  both  pitch  and  roll  axes.  The  vertical  gyro  OFF  flag  appeared 
on  the  ADI. 

(5)  Vertical  gyro  unreliable  failure  caused  the  vertical  gyro  to  slew  to 
a  slight  offset  (about  3-6  degrees  of  pitch  and  roll)  and  thereafter  operate 
in  a  non-linear  and  erratic  manner.  There  was  no  flag  representation  of 
this  failure. 
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(6)  Radar  altimeter  input  opened,  causing  the  radar  altimeter  display  to 
freeze  in  position.  Since  flare  mode  engagement  at  50  feet  was  triggered  by 
the  radar  altimeter,  the  flare  mode  did  not  engage.  The  radar  flag  appeared 
on  the  radar  altitude  instrument. 

The  display  flag-type  failures  were  grouped  according  to  location  and 
formed  principal  display  failure  variables.  Glideslope  and  vertical  gyro  dead 
failures  were  "ADI  failures."  Localizer  and  directional  gyro  failures  were 
"HSI  failures."  The  radar  altimeter  failure  represented  a  flag  failure  out¬ 
side  the  flight  director  system  displays.  The  vertical  gyro  unreliable  fail¬ 
ure  represented  an  insidious  failure  that  did  not  have  an  associated 
instrument  flag.  It  was  assumed  that  the  pilot's  ability  to  detect  flags 
within  any  one  of  these  three  locations  (ADI,  HSI,  and  RA)  would  be  similar. 
The  main  purpose  of  creating  different  failures  within  each  location  was  to 
vary  the  problem. 


Failure  Monitor  Modes 


The  amount  of  failure  monitor  information  presented  to  the  pilot  was 
varied  by  creating  three  failure  monitor  modes:  (l)  no  monitor,  (2)  caution, 
and  (3)  caution  plus  annunciator. 

No  monitor.  Basic  pilot  failure  detection  capability  was  determined  by 
pilot  performance  in  this  mode.  In  this  mode  only  existing  warning  flags  on 
the  Captain's  display  operated  during  those  failures  that  had  flag  repre¬ 
sentations.  The  vertical  gyro  unreliable  failure  did  not  cause  a  flag  to 
appear.  Control  system  failures  could  only  be  detected  by  inappropriate 
column  or  wheel  actions,  excessive  flight  director  command  errors,  or  an  atti¬ 
tude  and  path/course  following  discrepancy. 

Caution.  The  second  failure  monitor  mode  was  divided  into  two  categories 
in  order  to  test  the  utility  of  the  FLAG  light  on  the  fault  warning  mode  pro¬ 
gress  display.  One  half  of  the  subject  pilots  (Group  i)  received  a  flag 
light  for  flag  failures  and  a  master  caution  light  for  vertical  gyro  unreli¬ 
able  and  passive  and  softover  autopilot  control  failures .  Group  II  received 
a  caution  light  for  all  display  and  control  failures. 

Caution  plus  anuui  ciator.  The  third  failure  mode  brought  the  full  monitor 
and  annunciator  system  into  operation.  Control  system  failures  were  annunci¬ 
ated  by  the  automatic  control  annunciator  panel  (Figure  M-7)  in  addition  to 
the  master  caution  light.  Display  system  failures  were  annunciated  by  the 
system  annunciator  panel  (Figure  M-8),  master  caution  light  and  instrument 
warning  flags  (except  vertical  gyro  unreliable,  which  had  no  flag  representa¬ 
tion). 

Table  M-2  shows  the  system  operation  during  each  failure  mode. 


Display  Warning  Information  as  a  Function  of  Monitor  Mode 
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Winds 


In  addition  to  turbulent  air,  three  windshear  conditions  were  simulated. 
The  three  winds  were  a  tailwind  shear,  a  lateral  shear,  and  quartering  wind- 
shear  composed  of  both  tailwind  shear  and  lateral  shear  components.  The 
tailwind  shear  was  a  10-knot  steady-state  headwind  which  sheared  to  calm 
from  100  feet  to  touchdown.  The  lateral  shear  was  a  10-knot  steady- state  90- 
degree  crosswind  that  sheared  to  calm  from  100  feet  to  the  ground.  The 
quartering  windShear  was  a  vector  composed  of  both  tailwind  and  lateral  shear 
conditions  that  resulted  in  calm  winds  at  touchdown. 

Pilot  Task 

The  pilot '  s  task  was  to  fly  the  aircraft  to  a  successful  touchdown  using 
the  autopilot  and  flight  director  as  tools  to  accomplish  this  objective. 
Display  and  control  system  failures  could  be  expected.  Whenever  a  failure 
was  detected,  the  pilot  was  to  call  out  the  failure  and  take  corrective 
action.  If  a  control  failure  was  detected,  ohe  pilot  was  to  disconnect  the 
autopilot  axis  that  had  failed  and  continue  the  approach  by  flying  in  a  split- 
axis  mode.  If  a  display  failure  occurred,  the  pilot  was  to  continue  on 
autopilot  and  signify  his  decision  by  hitting  the  response  bar.  If  both  the 
autopilot  and  Captain's  display  failed  in  the  same  axis,  the  pilot  was  to 
execute  a  missed  approach  himself  or  relinquish  control  to  the  First  Officer 
as  the  situation  demanded.  At  the  initiation  of  a  missed  approach,  the 
pilot  was  to  engage  the  go-around  mode  of  the  flight  director  system. 

Requesting  the  pilots  to  continue  the  approach  after  experiencing  one 
failure  was  for  the  purpose  of  documenting  pilot  fault  detection  capability 
as  the  pilotfe  task  load  was  increased. 

For  those  approaches  that  ended  in  touchdown,  the  pilot  had  to  manually 
disengage  the  autopilot  at  touchdown,  deploy  the  spoilers,  reverse  thrust 
(the  F/0  would  usually  help  by  signifying  reverse  thrust  limit),  brake,  and 
steer  the  aircraft  down  the  runway  using  heading  and  raw  localizer  on  the 
HSI. 


Experimental  Designs 

The  primary  purpose  of  this  study  was  to  document  pilot  performance  with 
three  levels  of  failure  warning  information  during  control  and  display  system 
failures.  Following  any  one  of  these  failures,  or  combinations  thereof,  the 
pilots  either  executed  a  missed  approach  or  landed.  When  they  landed,  they 


did  so  under  different  split-axis  conditions  and  different  winds.  Pilot/sys¬ 
tem  performance  during  each  of  these  four  events  required  different  experi¬ 
mental  designs  which  follow: 

Control  failures.  Three  levels  of  fault  warning  information  (monitor 
modes)  were  tested  with  passive  and  softover  control  failures  in  pitch  and 
roll  for  each  of  the  nine  subjects  in  each  of  two  experimental  groups.  The 
design  is  diagrammed  in  Figure  M-9.  The  experimental  design  is  a  Lindquist 
Type  VI  (Ref.  5 ).  The  analysis  of  variance  source  tables  for  all  designs  are 
shown  in  Appendix  A . 


Failure  Axis 


FIGURE  M-9  -  Experimental  Design  for  Control  Failures 


Display  Failures,  Hie  pilot  was  asked  to  assume  split-axis  control 
following  a  control  system  failure  so  that  the  effect  of  pilot  task  load 
upon  display  failure  detection  could  be  examined.  Three  monitor  modes  were 
then  tested  with  four  display  failures  for  each  o~  the  nine  subjects  in  each 
group.  The  Lindquist  Type  VI  experimental  design  for  display  failures  is 
diagrammed  in  Figure  M-10.  Note  that  there  were  no  control  system  failures 
prior  to  display  failures  in  the  automatic  control  mode.  Control  system 
failures  were  assuned  to  put  the  pilot  into  split-axis  pitch  manual  and  split- 
axis  roll  manual  control  prior  to  the  display  failure  during  pitch  manual 
and  roll  manual  split-axis  conditions.  All  display  failures  during  split-axes 
control  were,  therefore,  second  failures. 


Pilot  Control  Task 


FIGURE  M-10  -  Experimental  Design  for  Display  Failures 
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Go-around  perform^*  -  Go-around  performance  was  placed  Into  a  mailer 
experimental  design  as  a  function  of  the  three  monitor  modes  and  the  two 
split-axis  control  conditions  prior  to  the  display  failure  that  caused  the 
pilot  to  execute  a  missed  approach.  Because  missed  approach  did  not  occur  an 
equal  number  of  times  within  each  condition,  a  randomised  block  design  and 
analysis  of  variance  was  used.  Data  from  all  missed  approaches  that  occurred 
under  the  conditions  defined  by  each  cell  in  Figure  M-ll  were  placed  into  that 
cell  and  formed  the  residual  error  term. 


Roll  Manual 


Fitch  Manual 


No  Monitor  Caution  Caution  + 

Annunciator 


Monitor  Mode 


FIGURE  M-ll  -  Experimental  Design  for  Go-Around  performance 


22 


The  effect  of  wind  upon  touchdown.  For  those  approaches  that  proceeded 
to  touchdown,  three  wind  conditions  were  confounded  with  failure  monitor 
modes.  The  effect  of  wind  upon  touchdown  performance  as  a  function  of  auto¬ 
matic,  pitch  manual  split-axis,  and  roll  manual  split-axis  control  was  of 
interest.  It  was  assumed  that  the  pilots  would  have  responded  to  all  control 
and  display  system  failures  prior  to  reaching  the  altitude  at  which  the  shear 
wind  started  taking  effect  (100  feet).  The  three  winds  were  a  tailwind  shear 
a  lateral  shear  and  a  quartering  shear  composed  of  both  tailwind  and  lateral 
shear  components.  The  three  winds  and  three  control  conditions  formed  the 
randomized  blocks  dr sign  shown  in  Figure  M-12.  There  was  an  unequal  number 
of  scores  in  each  cell  of  the  design  because  pilots  were  not  forced  to  con¬ 
tinue  to  touchdown;  the  decision  to  land  remained  the  prerogative  of  the 
pilot. 


Automatic  Pitch  Roll 

Manual  Manual 


r  ;GURE  M-12-  Experimental  Design  for  the  Effect  of  Wind  upon  Touchdown 
Performance 


Because  each  approach  was  flown  under  carefully  controlled  experimental 
conditions,  measured  changes  in  pilot  performance  can  be  attributed  to  the 
experimental  conditions.  The  measures  which  were  placed  into  each  experi¬ 
mental  design  for  analysis  appear  next. 
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Measurement 


Three  general  classes  of  measures  were  recorded  in  this  study:  pilot  pre¬ 
ference  measures,  pilot  detection  performance  measures  and  pilot/system  per¬ 
formance  measures. 

Pilot  preference  measures.  The  source  of  pilot  preference  data  was  the 
questionnaire.  After  all  flights  were  completed,  the  pilots  filled  out  a 
questionnaire  which  interrogated  their  opinions  of  the  primary  features  of 
the  fault  warning  system  in  terms  of  utility,  color  coding,  location  and  modes 
of  operation.  Comments  on  the  procedures  and  system  improvement  were  also 
solicited.  The  pilots  were  also  asked  to  scale  the  difficulty  of  detecting 
passive  autopilot  failures  during  each  of  the  three  monitor  modes. 

Pilot  detection  performance.  Each  time  a  failure  was  not  seen  by  a  subject 
pilot  within  30  seconds,  the  experimenter  serving  as  First  Officer  scored  a  no 
detection  for  that  condition.  An  incorrect  response  was  scored  when  the  pilot 
called  out  the  wrong  failure  or  identified  the  correct  failure,  but  his  action 
was  not  consistent  with  what  he  said.  (For  example,  an  incorrect  response 
would  be  scored  if  the  pilot  identified  a  pitch  axis  control  failure,  then 
proceeded  to  disconnect  the  roll  axis.)  A  false  alarm  was  scored  when  the 
pilot  called  out  a  failure  when  none  had  been  inserted.  The  pilot  was  never 
penalized  for  executing  a  go-around  when  he  could  have  continued  to  landing 
under  the  decision  logic  that  he  was  requested  to  use;  however,  an  incorrect 
decision  to  land  was  scored  when  the  pilots  continued  manually  to  touchdown 
when  there  was  a  display  failure  in  the  axis  that  they  were  controlling 
manually. 

Pilot /system  performance.  The  basic  data  consisted  of  10  channels  shown  in 
Table  M-3.  These  data  were  acquired  at  (l)  failure  insertion,  (2)  pilot 
response  and  (3)  touchdown  or  (4)  every  1.5  seconds  during  go-around.  These 
basic  data  were  further  treated  in  different  ways  to  describe  pilot/system 
performance  (l)  during  control  and  display  failures,  (2)  at  touchdown  and 
(3)  during  go-around. 

Failure  data  treatment.  Four  data  treatments  were  used  to  describe  pilot/ 
system  performance  during  control  and  display  failures.  The  first  treatment 
shown  in  Table  M-3  was  simply  the  value  of  the  indicated  parameter  at  pilot 
response.  The  second  was  the  absolute  value  (sign  ignored)  of  the  data  at 
pilot  response.  Describing  performance  using  absolute  values  is  most  impor¬ 
tant  in  lateral  plane  measures  because  the  system  tended  to  accrue  error  to 
one  side  or  the  other  as  a  function  of  the  steady-state  winds  which  were 
randomly  from  the  right  or  left.  The  absolute  value  describes  the  amount  of 
error  accrued,  independent  of  direction. 

The  third  way  of  treating  the  basic  data  was  to  find  the  change  in  system 
performance  between  failure  insertion  and  pilot  response.  The  change  in 
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TABUE  M-3 

Pilot /System  Performance  Measures 


Failure  Data  Treatments 


Basic  Data 
Channels 

Measurement  Pilot 
Accurate  to  Response 

Absolute  at 
Response 

Average 

Change 

Absolute 

Change 

Touchdown 

Data 

Longitudinal  l.o  feet 

Range* 

X 

X 

Lateral  u.5  feet 

Displacement 

X 

X 

X 

Pitch 

Attitude 

0.02  degrees  X 

X 

X 

X 

X 

Roll 

Attitude 

0.02  degrees  X 

X 

X 

X 

X 

Heading 
Error*  * 

0.02  degrees 

X 

X 

X 

Lateral 

Drift 

0.05  feet/sec.  X 

X 

X 

X 

X 

Absolute 

Altitude 

0.5  feet 

X 

Indicated 

Airspeed 

0.2  knots 

X 

X 

Vertical 

Velocity 

2.0  feet /min. 

X 

X 

Response 

Time 

0.02  sec. 

X 

♦From  Glideslope  transmitter. 

**  Heading  error  abcvt  nominal  248°  runway  compass  heading. 


25 


performance  v;as  generated  by  subtracting  the  values  of  the  indicated  channels 
at  pilot  response  from  the  value  at  failure  insertion.  The  fourth  data  treat¬ 
ment  subtracted  the  absolute  value  of  the  indicated  channels  at  pilot  response 
from  the  absolute  value  at  failure  insertion  in  order  to  assess  the  amount 
of  change  of  system  error  independent  of  direction. 

Touchdown  data  treatment.  The  values  of  the  eight  touchdown  parameters 
were  treated  with  regard  to  sign  (AE,  average  error)  and  without  regard  to 
sign  (AAE,  average  absolute  error). 

Go -around.  Go -around  was  considered  to  start  at  the  pilot  response  to  the 
second  failure.  When  go-around  mode  was  triggered  by  the  pilot,  data  were 
scanned  every  1.5  seconds  until  the  simulator  reset.  The  scan  that  contained 
lowest  altitude  was  used  in  conjunction  with  response  scans  and,  in  one  case, 
the  failure  insertion  scan  to  generate  go-around  data.  These  data  treatments 
are  listed  in  Tables  R-52  and  R-56  in  the  Go-around  performance  section. 


Procedures 

Pilot  briefing  and  training.  Upon  arrival  at  0800  hours,  each  pilot  was 
given  a  standardized  briefing  on  the  program,  the  specific  purpose  of  the 
study,  a  detailed  explanation  of  the  entire  system  operation  and  procedures 
to  be  followed.  Each  subject  pilot  was  then  seated  in  the  left-hand  seat  in 
the  simulator  cabin  and  was  briefed  again  by  the  experimenter /First  Officer. 
Four  familiarization  approaches  were  conducted.  All  system  failures  were 
exercised  to  provide  pilot  practice.  Two  approaches  terminated  in  touchdowns 
and  two  approaches  terminated  in  go-arounds.  After  practice,  39  approaches  for 
record  were  flown  (see  order  of  presentation,  below).  A  45-minute  lunch  break 
was  taken  at  a  logical  point.  Experimental  flying  continued  after  a  short 
re-briefing  and  warm-up  approach.  Experimental  flying  nominally  concluded  at 
1530  hours.  Following  the  flights  for  record,  the  subject  pilot  completed 
the  questionnaire,  was  de -brie fed  and  departed  the  facility  at  approximately 
1645  hours. 

Order  of  presentation.  The  39  approaches  were  divided  into  three  blocks 
of  13  approaches  each.  The  first  block  was  flown  with  no  monitor.  The  second 
block  was  flown  alternately  with  caution  light  failure  mode  for  one  subject, 
and  caution  plus  annunciator  for  the  next  subject.  The  last  block  of 
approaches  was  flown  using  the  untested  monitor  mode. 

Within  each  block  of  13  approaches,  one  trial  was  flown  automatic  until  the 
occurrence  of  a  flare  mode  failure  at  50  feet;  the  flare  mode  failure  approach 
was  randomly  placed  with  the  remaining  12  approaches.  These  remaining  12 
approaches  consisted  of  all  possible  combinations  of  four  display  failure 
locations  and  three  pilot  control  tasks,  randomly  ordered  for  each  block. 

The  pilot  control  tasks  at  the  display  failure  determined  the  control  failure 
axis  (pitch  or  roll);  no  control  failures  preceded  display  failures  during 
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"automatic ''  pilot  control  task.  Pitch  axis  control  failures  preceded  pitch 
manual  split-axis  display  failures  (there  were  four  in  each  block  of  13). 

Two  pitch  axis  control  failures  were  passive  and  two  were  softover  failures; 
therefore,  two  trials  of  each  failure  type  were  possible.  The  direction  of 
the  softover  failures  (up  or  down)  was  equally  distributed.  The  four  possi¬ 
bilities  (2  failure  types  x  2  trials)  within  the  axis  of  control  failure 
were  randomly  assigned.  Four  roll  axis  failures  leading  to  roll  manual 
split-axis  control  during  display  failires  were  similarly  treated. 

Failure  altitudes.  Failure  altitudes  were  randomly  assigned  within  an 
altitude  band  of  450  feet  to  100  feet  (except  the  flare  mode  failure  which 
had  to  occur  at  50  feet).  When  two  failures  were  given  on  one  approach,  the 
second  failure  had  to  be  programmed  to  occur  no  lower  than  100  feet,  and 
had  to  occur  no  sooner  than  20  seconds  after  the  preceding  control  failure. 
These  restrictions  were  imposed  because  we  wanted  the  pilot  to  have  ample 
time  to  detect  and  act  upon  the  first  failure  before  experiencing  the  second. 
We  also  wanted  the  second  failure  to  be  inserted  prior  to  the  windshear  at 
100  feet.  Previous  experience  indicated  that  20  seconds  would  be  more  than 
adequate. 

Wind  conditions.  Windshear  was  confounded  with  monitor  mode.  The  wind- 
shear  magnitude  and  type  (tailwind,  lateral,  or  quartering)  were  constant 
for  each  block  of  13  approaches.  The  direction  of  the  wind  was  randomly 
assigned  for  each  approach.  Having  assigned  the  wind  direction,  the  windshear 
was  set  to  shear  to  near  calm  on  the  ground  so  that  there  would  not  be  a 
requirement  to  decrab  the  aircraft.  The  console  operator,  acting  as 
Los  Angeles  approach  control,  reported  the  ground  winds  which  were  derived 
by  subtracting  the  shearwind  magnitude  and  vector  from  the  initial  condition 
lateral,  longitudinal  and  quartering  winds.  Turbulent  air  was  always  used. 
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RESULTS  AND  DISCUSSION 


Introductory  Comment 

General  organization  of  this  section.  The  purpose  of  this  study  was  to 
determine  pilot  fault  detection  capability  as  a  function  of  the  amount  of 
failure  monitor  information  that  was  present.  The  performance  data  were 
quite  dependent  upon  the  particular  failures  that  were  introduced.  There¬ 
fore,  a  major  portion  of  this  results  section  was  organized  according  to 
the  types  of  failures.  Control  failures,  display  failures  and  flare  mode 
failures,  go -around  performance,  the  effects  of  winds  upon  touchdown,  roll¬ 
out  performance,  and  a  general  discussion  of  fault-warning  and  mode-progress 
devices  are  discussed  in  this  section. 

Sub-section  organization.  Each  major  sub-section  starts  with  the  easiest 
data  form  to  interpret  and  proceeds  to  the  more  difficult.  The  number  of 
times  that  the  pilots  did  not  detect  a  failure  is  an  example  of  a  data  form 
that  appears  first.  Next,  pilot  response  times  and  aircraft  parameters  pro¬ 
vide  a  more  complex  look  at  pilot/system  performance.  Preference  (pilot 
opinion)  data  will  appear  as  applicable  within  each  sub-section.  Each  sub¬ 
section,  therefore,  presents  a  summarized  version  of  all  data  that  bears 
upon  the  topic  of  that  sub-section. 

Method  of  presenting  summarized  data.  Within  each  problem-oriented  sub¬ 
section,  some  data  are  shown  according  to  the  level  of  failure  monitor;  other 
data  are  shown  as  a  function  of  levels  of  control,  and  still  other  data  are 
reported  as  over-all  averages  and  standard  deviations.  The  decision  to 
summarize  or  break-out  (report  one  number  or  many  numbers)  performance  data 
is  based  upon  the  results  of  an  analysis -of -variance  test  where  one  was  con¬ 
ducted.  If  a  difference  between  means  ;.s  shewn  to  be  reliable  by  the 
analysis-of- variance  "decision  tool,"  then  the  means  are  independently 
reported.  For  example,  if  pilot  response  times  to  softover  control  failures 
were  different  from  their  response  times  to  passive  control  failures,  AND 
this  apparent  difference  was  shown  to  be  reliable  by  the  analysis  of  variance, 
then  both  means  would  be  shown.  If,  however,  the  statistical  tool  could  not 
confirm  a  significant  difference  (i.e.,  the  difference  could  have  occurred  by 
chance  alone)  then  the  data  would  simply  be  shown  as  one  average  response 
time  to  control  failures.  An  analysis -of- variance  summary  table  is  placed 
within  each  section. 

Finally,  it  is  quite  possible  for  a  source  of  variance  (  e.g.,  monitor 
mode)  to  be  significant  in  the  analysis  of  variance,  yet  some  of  the  means 
within  that  particular  source  may  not  be  significantly  different  from  each 
other.  For  example,  monitor  modes  may  be  a  significant  source  of  variance 
in  the  analysis  of  variance.  There  are  three  means  associated  with  this 
source,  one  for  each  level  (no  monitor,  caution  light  only,  and  the  full 
system).  The  distributions  of  scores  underlying  each  of  these  three  means 
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can  be  such  that  there  is  no  significant  difference  between  two  of  the  three 
means  even  though  the  tource  of  variance  was  significant.  When  a  source  of 
variance  is  significant,  the  Duncan's  New  Multiple  Range  Test  (Ref.  l)  is 
used  to  find  out  which  Leans  are  significantly  different.  Duncan's  test 
results  are  indicated  either  by  footnotes  to  the  tables  of  means  and  standard 
deviations  or  by  placing  boxes  around  the  means  which  are  not  significantly 
different.  Since  the  tables  of  means  also  contain  standard  deviations,  the 
boxes  are  placed  around  means  and  standard  deviations  even  though  the  box 
refers  to  the  means.  The  interpretation  to  be  used  is  that  numbers  within 
the  same  box  are  not  significantly  different. 


Control  Failures 


Detection.  Out  of  216  passive  control  failures  that  were  administered,  39 
were  undetected.  Five  out  of  2l6  softover  control  failures  were  undetected. 
Table  R-l  shows  that  the  number  of  times  that  failures  were  undetected  reduced 
as  the  amount  of  monitor  system  information  increased.  Softover  failures  did 
not  seem  to  present  a  detection  problem,  but  passive  failures  were  difficult 
to  detect.  The  full  monitor  system  with  caution  light  ind  annunciator  panel 


TABLE  R-l 

Control  Failure  Detection  as  a  Function  of  Monitor  Modes 


Softover  Failure  Passive  Failure 


Measure 

No 

Monitor 

With 

Caution 

Caution 
+  Annun. 

No 

Monitor 

With 

Caution 

Caution 
+  Annun, 

Number  of  times  fail 
ure  not  detected 
* 

2 

2.8* 

2 

2.8* 

1.4* 

21 

29.2* 

16 

22.2* 

2 

2.8* 

Number  of  incorrect 
responses* 

* 

2 

2.8% 

1.4* 

0 

0.0* 

3 

4.2* 

4 

5.6* 

2 

2.8* 

Number  of  false 
alarms** 

* 

1 

1.4* 

1 

1.4* 

0 

0.0* 

2 

2.8* 

4 

5.6* 

1 

1.4* 

♦Incorrect  responses  include  incorrect  identification  of  failure. 

**A  false  alarm  was  scored  when  a  pilot  called  out  a  failure  that  didn't 
exist.  Note  that  most  false  alarms  were  thought  to  be  passive 
autopilot  failures. 
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was  necessary  to  reduce  the  non-detections  of  passive  failures  to  two  times 
out  of  72  failures .  Table  R-2  shows  that  the  pitch  axis  passive  failure  was 
most  difficult  to  detect.  Contrary  to  what  was  seen  in  passive  failures, 
the  roll  axis  soft over  failure  appeared  more  difficult  to  detect  than  the 
pitch  axis  softover  failure. 

Incorrect  responses.  An  incorrect  response  was  scored  when  the  pilot 
either  (1)  called  out  a  pitch  failure  instead  of  a  roll  failure  (or  the 
reverse),  or  (2)  correctly  identified  the  failed  axis  but  disengaged  the 
wrong  axis,  or  (3)  disengaged  the  whole  autopilot.  Table  R-l  shows  that 
the  number  of  incorrect  responses  reduced  when  the  full  monitor  system  was 
used.  Passive  failures  again  were  the  problem.  The  two  incorrect  respon¬ 
ses  shown  (Table  R-l)  for  passive  failures  with  the  caution  plus  annunciator 
were  due  to  incorrect  identifications  of  the  failure  even  though  the 
responses  were,  in  fact,  correct.  Table  R-2  indicates  that  the  roll  axis 
passive  failure  accounted  for  most  of  the  incorrect  responses:  These 
incorrect  responses  were  largely  caused  by  confusion  between  the  passive 
autopilot  failure  and  the  vertical  gyro  unreliable  failure. 


TABLE  R-2 

Control  Failure  Detection  as  a  Function  of  Failed  Axis 


Softover  Failure 

Passive  Failure 

Measure 

Pitch 

Manual 

Roll 

Manual 

Pitch 

Manual 

Roll 

Manual 

Number  of  times 

failure  not  detected 

1 

k 

22 

17 

$ 

0.9 $ 

3.7% 

20.4$ 

15.8$ 

Number  of  incorrect 

responses* 

1 

2 

1 

8 

$ 

0.9$ 

1.9 % 

0.9 % 

7.4$ 

Number  of  false  alarms 

1 

1 

7 

0 

$ 

0.9$ 

0.9% 

6.5$ 

0.0$ 

♦Incorrect  responses  include  incorrect  identification  of  failure. 
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False  alarms.  Although  the  numbers  were  not  high,  more  false  alarms 
appeared  in  the  monitor -with -caution -conditions  (Table  R-l)  than  without  any 
monitor  or  with  the  caution  light  and  annunciator  panel.  Table  R-2  shows 
that  the  false  alarms  were  thought  to  be  pitch  passive  failures. 

Summary  of  detection,  incorrect  responses  false  alarms.  Passive  fail- 
ure  non-detections,  incorrect  responses  and  false  alarms  together  (Table  R-l) 
indicated  classic  human  operator  signal  detection  behavior  in  the  presence 
of  uncertainty  and  noise.  Hie  number  of  non-detections  decreased  when  the 
caution  light  was  added;  but*  the  number  of  incorrect  responses  and  the  false 
alarms  tended  to  increase.  One  suspects,  therefore,  that  guessing  was  going 
on — the  pilots  were  catching  more  failures  in  the  process  of  guessing  but 
introducing  "failures"  of  their  own  by  either  incorrectly  responding  or 
calling-out  a  failure  that  did  not  exist.  On  the  basis  of  these  data  and 
criteria,  the  passive  control  failure  was  shown  to  be  the  detection  problem 
during  no  monitor  and  caution  conditions.  When  the  annunciator  was  added, 
there  was  a  marked  improvement  in  fault  detection  and  response  performance. 

Response  time.  Response  time  indicated  the  time  that  it  took  the  pilots 
to  detect  a  control  failure,  call  out  the  problem  axis  and  disengage  the 
autopilot  in  the  failed  axis.  Table  R-3  shows  average  pilot  response  times 
that  resulted  from  soft  over  and  passive  control  failures  for  each  of  the 
monitor  modes. 


TABI£  R-3 

Pilot  Response  Time  to  Control  Failures 


Monitor  Mode 

Across 

Failure  No  With  Caution  Monitor 

Intensity  Monitor  Caution  +  Annunciator  Mode 


Passive 


So ft over 


15.68* 

14.34 

8.43 

(3.31)** 

(4.40) 

(0.68) 

7.15 

6.82 

7.18 

(2.08) 

(1.98) 

(1.32) 

12.82 

(4.48) 


7.05 

(1.82) 


Across  11.41  10.58  7.81  9.93 

Failure  (5.10)  (5.08)  (1.22)  (4.47) 

Intensity 


*  Mean  response  time  in  seconds. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly 
different . 
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It  took  much  longer  for  the  pilots  to  respond  to  passive  failures  than 
to  softover  failures  when  no  monitor  was  present  (15.68  sec.  vs.  7.15  sec.). 
The  addition  of  a  monitor  with  a  caution  light  did  not  significantly  change 
response  time  from  the  no  -monitor  condition.  When  the  annunciator  panel  was 
operating,  pilot  response  time  to  passive  failures  reduced  to  8.43  sec., 
which  was  not  significantly  different  from  their  response  times  to  softover 
f allures . 

Disregarding  monitor  modes,  pilot  responses  to  passive  failures  took 
longer  than  responses  to  softover  failures  (Table  R-3)  and  it  took  an 
equivalent  amount  of  time  to  respond  to  either  pitch  axis  or  roll  axir 
passive  failures.  Softover  failures,  on  the  other  hand,  were  axis  dependent. 
Pilots  responded  to  pitch  softover  failures  (5.94  sec.)  more  quickly  than 
they  responded  to  roll  softover  failures  (8.16  sec.). 


TABLE  R-4 

Pilot  Response  Time  to  Control  Failures 


Failure 

Intensity 

Failed  Axis 

Pitch 

Roll 

Across 

Axis 

Failed 

Passive 

12.63* 

13.01 

12.82 

(4.44)** 

(4.56) 

(4.58) 

Softover 

5.94 

8.16 

7.05 

(1.27) 

(1.59) 

(1.82) 

Across 

Failure 

9.28 

10.58 

9.93 

Intensity 

(4.67) 

(4.18) 

(4.47) 

*  Mean  response  time  in  seconds, 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 


Summary  of  response  time.  Passive  failures  appeared  to  be  more  diffi¬ 
cult  to  detect  in  the  no-monitor  and  monitor-with-caution-light-only 
conditions.  It  took  just  as  long  for  pilots  to  detect  either  pitch  or 
roll  passive  failures .  Pitch  softover  failures  were  detected  more  readily 
than  roll  softover  failures.  Wi.th  the  monitor  and  annunciator  panel 
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operating, passive  and  active  failures  were  equally  detected,  but  the  pilot 
response  time  was  still  about  7.0  sec.  Under  the  conditions  of  the  study,  the 
monitor  took  5.0  seconds  to  detect  the  failure.  One  could  thus  assume  the 
pilot  response  time  to  the  monitor  to  be  2.0  seconds. 

Altitude  used.  Altitude  used  measured  the  amount  of  altitude  consumed  from 
failure  insertion  until  pilot  response.  This  measure  indicated  the  altitude 
consequences  of  the  response  times  just  presented. 

Essentially  the  same  pattern  indicated  in  response  time  measures  emerged. 
Table  R-5  shows  the  altitude  that  was  consumed  as  a  consequence  of  each  combi¬ 
nation  of  monitor  mode  and  failure  intensity.  The  monitor  levels  examined  had 
no  effect  upon  the  amount  of  altitude  used  during  soft over  failures.  Hie 
monitor -with- caution  -light  did  not  alter  the  altitude  used  during  passive  fail¬ 
ures  when  compared  to  the  altitude  used  when  there  was  no-monitor  at  all.  In 
fact,  the  addition  of  the  caution  light  increased  the  variability  (in  terms  of 
one  standard  deviation)  of  altitude  used  during  passive  failures.  The  monitor 
with  annunciator  panel  reduced  the  passive  failure  altitude  consumption  to  the 
softover  failure  level.  When  the  monitor  with  caution  and  annunciator  opera¬ 
ted,  there  was  an  average  altitude  consumption  of  74  feet  with  a  13.7-foot 
standard  deviation. 


TABLE  R-5 

Altitude  Used  During  Control  Failures 


Failure 

Intensity 

Monitor  Mode 

Across 

No 

Monitor 

With 

Caution 

Caution  Monitor 

+  Annunciator  Mode 

Passive 

146.40* 

144.44 

80.26 

123.70 

(30.94)** 

(47.96) 

(10.71) 

(45.34) 

Softover 

68.47 

69.86 

68.26 

68.86 

(18.61) 

(21.89) 

(13.85) 

(18.26) 

Across 

Failure 

107.43 

107.15 

74.26 

96.28 

Intensity 

(46.72) 

(52.73) 

(13.70) 

(44.10) 

♦Mean  altitude  used  in  feet. 

**0ne  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly 
different. 
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There  was  a  difference  in  the  amount  of  altitude  used  as  a  function  of 
failure  axis  (Table  R-6).  More  altitude  was  used  during  roll  axis  failures, 
probably  because  the  pilot  response  time  to  roll  failures  was  longer  than 
response  time  to  pitch  failures.  Although  the  previous  pilot  response  time 
measures  indicated  i  difference  between  pitch  and  roll  axis  failures  as  a 
function  of  failure  intensity  (softover  or  passive— Table  R-4),  the  time 
differences  reported  had  no  reliable  effect  upon  the  altitude  consumed. 


TABI£  R-6 

Altitude  Used  During  Control  Failures 


Failed  Axis 


Across 

Failed 

Measure 

Fitch 

Roll 

Axis 

Average 

89.64* 

102.92 

96.28 

Standard  Deviation 

44,'fc** 

42.62 

44.10 

*  Feeti 

**  One  standard  deviation. 


Changes  in  the  amount  of  altitude  used  can  be  related  to  pitch  attitude 
changes  and  vertical  velocity  changes  as  well  as  response  time.  As  aircraft 
parameters  that  result  in  altitude  change,  pitch  attitude  and  vertical 
velocity  will  be  examined  next. 

Pitch  attitude.  The  actual  data  treatment  being  reported  here  took  the 
absolute  value  (signs  ignored)  of  pitch  attitude  at  failure  insertion  and 
subtracted  from  it  the  absolute  value  of  the  pitch  attitude  at  pilot  response. 
Since  pitch  attitude  normally  was  about  2.0°  at  failure  insertion,  a 
larger  (i.e.,  3°)  pitch  attitude  at  response  shows  as  a  negative  number  in 
these  data.  This  data  treatment  was  chosen  to  represent  pitch  attitude 
because  it  was  the  most  sensitive. 
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Table  R-7  indicates  that  during  pitch  softover  failures  a  1.32 -degree 
pitch-up  tendency  occurred.  Since  pitch-up  and  pitch -down  softover  failures 
occurred  an  equal  number  of  times,  it  is  likely  that  the  pilots  were  more 
sensitive  to  the  occurrence  of  a  pitch-down  than  a  pitch-up, and  tended  to 
restrain  the  column  pitch-down. 


TABU)  R-7 

Pitch  Attitude  Change  During  Control  Failure 


Failure 

Intensity 

Failed  Axis 

Pitch 

Roll 

Across 

Axis 

Failed 

Passive 

-0.24* 

-0.03 

-0.14 

(0.82)** 

(0.42) 

(0.66) 

Softover 

-1.32 

0.18 

-0.57 

(1.80) 

(0.57) 

(1.53) 

Across 

Failure 

"0.78 

0.08 

-0.35 

Intensity 

(1.49) 

(0.51) 

(1.19) 

*  Mean  pitch  attitude  change  in  degrees.  Note  that  a  negative 
implies  response  value  larger  than  failure  insertion  (pitch-up). 
**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly 
different . 
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It  is  evident  that  pitch  attitude  simply  did  not  change  as  a  function  of 
monitor  mode  (Table  R-8)  as  did  the  response  time  or  altitude  used.  The 
altitude  consumed  that  was  previously  shown  cannot  be  due  to  any  changes  in 
pitch  attitude  as  a  function  of  men i tor  mode.  Finally,  there  was  no  evi¬ 
dence  of  roll  control  failure  getting  so  far  out  of  hand  as  to  cross-couple 
into  pitch. 


TABLE  R-8 

Pitch  Attitude  Change  During  Control  Failure 


Monitor  Mode 


Across 

Failure  No  tilth  Caution  Monitor 

Intensity  Monitor  Caution  +  Annunciator  Mode 


Passive 


Soft  over 


0.06  * 

(0.73)** 

-0.36 

(0.68) 

-0.11 

(0.49) 

-0.62 

(1.37) 

-0.36 

(1.25) 

-0.72 

(1.91) 

-0.14 

(0.66) 

-0.57 

(1.53) 


Across 

Failure  -0.28  -O.36  -0.42  -0.35 

Intensity  (l.l4)  (1.00)  (1.42)  (1.19) 


*  Mean  pitch  attitude  change  in  degrees;  negative  implies  pitch-up. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 


Vertical  velocity  change.  This  measure  was  the  amount  of  change  between 
vertical  velocity  at  failure  insertion  and  vertical  velocity  when  the  pilot 
responded  to  the  control  failure.  Unlike  the  attitude  change  measure, 
vertical  velocity  retained  the  original  signs  attached  to  the  values.  Since 
vertical  velocity  at  failure  insertion  was  nominally  -640  feet  per  minute, 
a  decrease  in  vertical  velocity  at  pilot  response  (e.g..  -440  fpm. )  would 
show  as  a  negative  number  {?.g, ,  -640-(  -440)  =  -20QJ .  Therefore, 

negative  vertical  velocity  change  values  show  a  lessening  of  sink  rate. 
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Table  R-9  shews  that  vertical  velocity  tended  to  have  increased  by  the  time 
pilots  responded  to  pitch  failures  with  no  monitor.  Vertical  velocity  had 
decreased  at  pilot  response  during  pitch  failures  in  the  monitor  with  caution- 
light-only  approaches.  The  tendency  of  the  pilots  to  decrease  vertical  velo¬ 
city  during  the  caution-light-only  pitch  failures  was  interesting.  Apparently, 
the  pilots  were  holding  a  slight  pitch-up  bias  while  they  were  diagnosing  the 
failure.  This  tendency  was  not  seen  during  other  conditions,  and  was  not 
clearly  evident  in  the  pitch  attitude  measure.  Hie  value,  -77.3  feet  per 
minuce  of  change  between  failure  insertion  and  pilot  response,  was  rather 
small;  the  effect,  therefore,  was  not  profound.  Finally,  when  the  full  moni¬ 
tor  system  with  caution  light  and  annunciator  panel  was  operating,  there  were 
no  differences  in  vertical  velocity  as  a  function  of  pitch  or  roll  control 
failures.  Vertical  velocity  data  variability  was  twice  as  high  during  pitch 
failures  as  during  roll  failures,  as  would  be  expected. 


TABI£  R-9 

Vertical  Velocity  Change  During  Control  Failures 


Monitor  Mode 


Across 


Axis 

No 

With 

Caution 

Monitor 

Failed 

Monitor 

Caution 

+  Annunciator  Mode 

Pitch 

62.3* 

-77.3 

19.1 

1.4 

(181.3)** 

(183.5) 

(192.6) 

(193.2) 

Roll 

-8.5 

0.0 

-14.3 

-7.6 

1 

(96.3) 

(*7.1) 

(79.2) 

(87.2) 

Across 

Axis 

26.9 

-38.7 

2.4 

-3.1 

Failed 

(148.5) 

(147.5) 

(147.2) 

(149.6) 

*  Mean  vertical  velocity  change  in  feet  per  minute;  negative  implies 
pitch-up. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly 
different. 


Disregarding  monitor  mode,  Table  R-10  shows  that  vertical  velocity  at 
pilot  response  during  passive  failures  had  decreased,  while  vertical  velo¬ 
city  at  pilot  response  for  softover  failures  was  not  significantly  different 
from  the  vertical  velocity  change  during  roll  failures.  In  this  case  the 
data  were  felt  to  be  quite  sensitive  to  a  mechanization  problem. 

There  was  a  slight  bias  in  the  pitch  axis  trim  mechanisation  of  the  auto¬ 
pilot  in  the  simulator.  Pitch  trim  follow-up  was  designed  to  aerodynami- 
cally  trim  servo  load.  As  in  any  feedback  control  sjstem,  the  system  seeks 
equilibrium,  but  cannot  achieve  perfect  equilibrium.  The  mechanization  was 
such  that  a  very  slight  pitch-up  aerodynamic  trim  remained  that  was  well 
within  the  requirements  for  the  autopilot  to  leave  the  aircraft  in  a  trim 
condition  upon  disconnect.  When  the  autopilot  was  failed  passive,  the  servo 
released  the  elevator  and  the  trim  bias  that  existed  produced  a  pitch-up 
tendency  which  resulted  in  an  average  50-foot-per-minute  change  in  vertical 
velocity  at  the  end  of  an  average  12  seconds  of  time.  Since  the  softover 
control  failure  did  not  release  the  elevator  from  the  servo  until  the  moment 
of  pilot  response,  this  effect  was  not  seen. 


TABLE  R-10 

Vertical  Velocity  Change  During  Control  Failures 


Axis 

Failed 

Failure 

Passive 

Intensity 

Softover 

Across 

Failure 

Intensity 

Pitch 

-56.5* 

59.2 

1.4 

(174.9)** 

(194.8) 

(193.2) 

Ron 

-6.2 

-9.0 

-7.6 

(90.4) 

(84.6) 

(87.2) 

Across 

Failed 

-34.0 

25.1 

-3.1 

Axis 

(140.9) 

(153.4) 

(149.6) 

*  Mean  vertical  velodfcy  change  in  feet  per  minute;  negative  implies  pitch-up. 
**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 
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Longitudinal  range.  The  change  between  longitudinal  range  at  failure 
insertion  and  longitudinal  range  at  pilot  response  showed  the  distance  that 
the  vehicle  traveled  while  the  pilot  was  detecting  and  acting  on  the  failure. 
Table  R-ll  diows  that  distance  traveled  varied  with  monitor  mode  and 
failure  intensity  in  the  same  way  that  response  time  and  altitude  used  did. 
About  3,127  feet  were  traveled  during  an  average  passive  failure  with  no 
monitor  and  monitor  with  caution  light.  Since  the  remaining  means  were  not 
significantly  different,  the  average  of  these  scores,  1,521  feet,  describes 
range  consumed  during  all  other  conditions. 


TABI£  R-ll 

Longitudinal  Range  Consumed  During  Control  Failures 


Monitor  Mode 


Across 


Failure 

Intensity 

No 

Monitor 

With 

Caution 

Caution  Monitor 

+  Annunciator  Mode 

Passive 

3133* 

(610)** 

3121 

(968) 

1680 

(183) 

2645 

(953) 

Soft  over 

1 1439 
! (438) 

1480 

(425) 

1485 

(253) 

1451 

(378) 

Across 

Failure 

2286 

2300 

1557 

2048 

Intensity 

(1003) 

(mi) 

(252) 

(939) 

*  Mean  longitudinal  range  in  feet. 

**  Cte  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 
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Table  R-12  showing  distance  traveled  as  a  function  of  failure  intensity 
and  failed  axis  corroborates  the  response  time  values  of  the  same  inter¬ 
action  (axis  failed  by  failure  intensity)  shown  in  Table  R-4. 

t  ■ 

TABLE  R-12 

Longitudinal  Range  Consumed  During  Control  Failures 


Axis  Failed 


Failure 

Intensity 

Pitch 

Roll 

Across 

Axis 

Failed 

Passive 

2607* 

2682 

2645 

: 

(939)** 

(974) 

(953) 

|  > 

•  Softover 

1221 

1681 

1451 

1 

(244) 

(349) 

(378) 

Across 

1914 

2181 

2048 

Failure 

Intensity 

(975) 

(885) 

(939) 

*  Mean  longitudinal  range  in  feet. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 


Indicated  airspeed.  The  average  change  in  indicated  airspeed  from  fail¬ 
ure  insertion  until  pilot  response  was  homogeneous  for  all  failures.  On  the 
average,  O.85  knot  was  lost  with  a  standard  deviation  of  2.19  knots.  The 
autothrottle  system  installed  in  the  simulator  held  airspeed  constant  through 
the  control  failures  that  were  inserted. 

Lateral  displacement.  The  change  in  absolute  lateral  displacement  was 
determined  by  subtracting  the  absolute  value  of  lateral  displacement  at 
pilot  response  from  the  absolute  value  of  lateral  displacement  at  failure 
insertion.  Negative  values  indicate  that  displacement  was  larger  at  pilot 
response  than  at  failure  insertion.  The  amount  of  lateral  displacement  was 
dependent  upon  each  combination  of  failure  intensity,  monitor  mode^  and  axis. 
During  pitch  axis  failures,  there  were  no  differences  in  lateral  displacement 
as  a  function  of  monitor  mode  or  failure  intensity  as  one  might  expect. 

The  average  lateral  displacement  change  during  all  pitch  axis  failures  was 
1.28  feet  with  a  standard  deviation  of  5.17  feet.  Table  R-13  shows  the 
lateral  displacement  changes  which  occurred  during  roll  failures. 

4l 
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TABLE  R-13 

Lateral  Displacement  Change  During  Roll  Control  Failures 


Monitor  Mode 


Across 


Failure 

No 

With 

Caution 

Monitor 

Intensity 

Monitor 

Caution 

+  Annunciator  Mode 

Passive 

-29.92* 

-21.61 

-1.10 

-17.55 

(28.45)** 

(22.46) 

(7.55) 

(24.28) 

Softover 

-15.10 

-16.62 

-9.33  1 

-13.69 

(17.35) 

(21.09) 

l(20.20)| 

(19.50) 

Across 

-22.52 

_ 

-19.12 

_ 

-5.22 

-15.62 

Failure 

(24.41) 

(21.62) 

(15.60) 

(22.00) 

Intensity _ 

*  Mean  lateral  displacement  in  feet. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  boot  are  not  significantly  different. 


In  the  no-monitor  case,  lateral  displacement  during  passive  failures  changed 
almost  30  feet,  whereas  during  softover  failures,  the  lateral  displacement 
changed  only  15  feet.  Note  the  variability  underlying  these  means  in  Table 
R-13.  With  no  monitor  during  a  passive  failure,  68^  of  the  scores  were  between 
zero  and  sixty  feet.  When  the  monitor-with-caut ion-light  was  operating,  the 
difference  between  passive  and  softover  failures  became  non-significant. 

Average  lateral  displacement  during  the  caution  light  monitor  mode  was,  there¬ 
fore,  19.12  feet  with  a  standard  deviation  of  21.62  feet. 

When  the  monitor  with  caution  and  annunciator  panel  operated,  performance 
improved  considerably.  Again,  since  there  was  no  significant  difference 
between  the  means  indicated,  the  average  lateral  displacement  change  with  the 
full  system  was  5.22  feet. 

Considering  the  softover  failures  by  themselves,  monitor  mode  did  not 
influence  the  amount  of  lateral  displacement  change  from  failure  insertion 
until  pilot  response. 

Before  concluding  the  discussion  of  lateral  displacement,  it  would  be  well 
to  examine  roll  attitude,  heading  error  and  drift  rates  that  lead  to  the 
lateral  displacements  that  were  found. 
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Roll  attitude.  This  measure  indicated  the  absolute  amount  of  roll  change 
from  failure  insertion  to  pilot  response  in  a  maimer  similar  to  lateral 
displacement.  The  amount  of  roll  attitude  change  was  a  function  of  either 
(l)  monitor  mode,  or  (2)  failure  intensity  for  roll  failures.  Failure  inten¬ 
sity  did  not  interact  with  monitor  mode  as  in  the  lateral  displacement 
measure. 

The  amount  of  roll  attitude  change  from  failure  insertion  until  pilot 
response  was  constant  until  the  caution  light  and  annunciator  panel  were  in 
operation  (Table  R-l4).  When  the  full  system  was  in  operation,  there  was  a 
reduction  in  roll  attitude  change. 


TABLE  R-l4 

Roll  Attitude  Change  During  Control  Failures 


Monitor  Mode 


Across 

No 

With 

Caution 

Monitor 

Measure 

Monitor 

Caution  +  Annunciator 

Mode 

Mean 

-3.23* 

-3.26 

-2.54 

-3-01 

Standard 

Deviat  ion 

4.24** 

4.21 

3.74 

4.06 

*  Mean  roll  attitude  in  degrees. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 


Independent  of  monitor  mode,  passive  roll  failures  yielded  an  average  of 
3.17  degrees  of  roll  attitude  change  at  pilot  response  whereas  soft over 
failures  led  to  an  average  8.8  degrees  of  roll  change  (Table  R-15).  Even 
though  pilot  response  times  to  softover  failures  were  quite  faster  than 
response  times  to  passive  failures,  higher  roll  attitudes  prevailed  as  a 
consequence  of  the  softover  failures.  These  higher  roll  attitudes  did  not 
regularly  lead  to  greater  lateral  displacement  (refer  again  to  Table  R-13), 
because  softover  failures  resulted  in  generally  lower  lateral  displacements 
than  did  passive  failures.  Neither  did  the  roll  attitude  changes  during 
passive  failures  lead  to  more  heading  error. 
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TABU)  R-15 

Roll  Attitude  Change  During  Control  Failures 


Failure 

Intensity 

Failed  Axis 

Across  Failed 
Axis 

Pitch 

Roll 

Passive 

-0.01* 

-3.17 

-1.59 

(0.37)** 

(2.77) 

(4.81) 

So  ft over 

-0.06 

-8.80 

-4.43 

Across 

(0.12) 

(2.61) 

(2.43) 

Failure 

-0.03 

-5.99 

-3.01 

Intensity 

(0.28) 

(3.89) 

(4.06) 

*  Mean  roll  altitude  change  in  degrees. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 


Heading  error.  Heading  error  was  the  difference  between  the  aircraft 
heading  and  the  runway  heading  of  248  degrees.  The  measure  reported  here 
was  the  absolute  change  in  heading  error  between  failure  insertion  and  pilot 
response.  Roll  axis  failures  (Table  R-l6)  created  heading  error  changes  until 
the  full  monitor  system  with  annunciator  panel  was  operating.  With  the  full 
system  in  operation,  there  were  no  differences  between  heading  error  change 
shown  for  roll  failures  and  for  pitch  failures. 
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TABLE  R-l6 

Heading  Error  Changes  During  Control  Failures 


Monitor  Mode 


Failed 

No 

With 

Caution 

Axis 

Monitor 

Caution 

+  Annunciator 

Pitch 

-0.06* 

0.04 

-0.08 

(0.54)*+ 

(0.50) 

(0.53) 

Roll 

-0.83 

— 

-0.45 

-0.09 

(1.32) 

— 

(1.29) 

(0.84) 

Across  -0.44  -0.21  -0.09 
Axis  (1.07)  (1.00)  (0.70) 
Failed 


Across 

Monitor 

Mode 

-0.04 

(0.52) 


-0.1*6 

(1.20) 


-0.25 

(0.94) 


*  Mean  heading  error  change  in  degrees. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 


Contrary  to  what  was  shown  for  roll  attitude  (Table  R-15),  heading  error 
indicated  in  Table  R-l?  was  greater  as  a  consequence  of  passive  failures. 

Even  though  roll  attitude  changes  at  pilot  response  were  high  during  softover 
failures,  pilot  response  tine  was  faster  which  resulted  in  less  change  in 
heading  error  and,  consequently,  less  lateral  displacement. 
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TABLE  R-17 

Heading  Error  Change  During  Control  Failures 


Failed  Axis 


Failure 

Intensity 

Pitch 

Roll 

Across 

Axis 

Failed 

Passive 

-0.07* 

-0.74 

-0.40 

(0.57)** 

(1.34) 

(1.08) 

Softover 

-0.00 

-0.18 

-0.09 

(0.47) 

(0.96) 

(0.76) 

Across 

-0.04 

-0.46 

-0.25 

Failure 

Intensity 

(0.52) 

(1.20) 

(0.94) 

*  Mean  heading  error  change  in  degrees. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 


Lateral  drift.  Lateral  drift  difference  was  treated  the  same  as  the  pre¬ 
vious  measures .  The  change  in  lateral  drift  was  greater  during  roll  failures 
than  during  pitch  failures,  as  expected.  During  roll  failures,  adding  a 
caution  light  to  the  system  did  not  significantly  change  lateral  drift 
(Table  R-l8).  When  the  caution  light  and  annunciator  panel  were  both  opera¬ 
ting,  the  change  in  lateral  drift  reduced  to  6.47  feet  per  second.  Previous 
measures  of  heading  error,  roll  attitude  and  lateral  displacement  indicated 
that  pitch  and  roll  axis  failures  were  equivalent  in  the  caution  plus  annunci¬ 
ator  panel  monitor  mode.  This  was  not  the  case  for  lateral  drift,  because 
roll  axis  failures  produced  lateral  drift  values  that  were  significantly 
different  from  those  attained  during  pitch  axis  failures  in  the  caution  plus 
annunciator  mode. 


TABLE  R-l8 

Lateral  Drift  Change  During  Control  Failures 


Monitor  Mode 


Across 


Failed 

No 

With 

Caution 

Monitor 

Axis 

Monitor 

Caution 

+  Annunciator 

Mode 

Pitch 

-0.04* 

(0.68)** 

-0.21 

(1.08) 

-0.18  1 
(0.92)  i 

-o.i4 

(0.90) 

Roll 


WO.  39  -9.81  -6.47 

1  (6.49)  (5.73)  (5.16) 


-8.89 

(6.02) 


Across 

Axis 

Failed 


-5.21 

(6.94) 


-5.01 

(6.34) 


-3.32 

(4.85) 


-4.51 

(6.14) 


*  Mean  lateral  drift  in  feet  per  second. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 


Looking  further  into  the  caution  plus  annunciator  mode,  it  was  found 
that  lateral  drift  was  also  a  function  of  failure  intensity  in  that  mode. 
Table  R-19  shows  that  softover  failures  produce  the  higher  drift  in  the 
caution  plus  annunciator  mode. 


TABLE  R-19 

Lateral  Drift  Change  During 

Control  Failures  in  the  Caution  +  Annunciator  Monitor  Mode 


Failure 

Standard 

Intensity 

Mean 

Deviation 

Passive 

-1.4l* 

2.46 

Softover 

-5.24 

5.85 

*  Feet  per  second. 
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Disregarding  monitor  mode,  roll  axis  soft overs  (Table  R-20)  created  lateral 
rates  of  11.04  feet  per  second,  whereas  passive  failures  showed  6.74  feet  per 
second  lateral  rates  at  pilot  response.  These  lateral  drift  changes  verily 
the  consequence  of  roll  attitude  changes  seen  earlier. 


TABLE  R-20 

Lateral  Drift  Change  During  Control  Failures 


Failed  Axis 


Failure 

Intensity 

Pitch 

Roll 

Across 

Axis 

Failed 

Passive 

-0.34* 

-6.74 

-3.54 

(0.90  )** 

(5.73) 

(5.20) 

Softover 

-0.06 

-11.04 

-5.49 

(0.86) 

(5.55) 

(6.83) 

Across 

-8.89 

-4.51 

Failure 

-0.14 

Intensity 

(0.90) 

(6.02) 

(6.14) 

*  Mean  lateral  drift  in  feet  per  second. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 


Sumnary  of  lateral  displacement  ehanuraa.  The  effect  of  failure  monitor 
mode  upon  lateral  plane  performance  was  generally  the  same  as  the  vertical 
plane.  Generally,  there  were  no  differences  between  the  no  monitor  and  the 
caution  light  only;  but,  performance  improved  with  the  addition  of  the 
annunciator  panel.  Softover  failures  led  to  high  roll  attitude  and  lateral 
drift  at  pilot  response;  but,  possibly  because  response  times  to  softover 
failures  were  considerably  faster  than  response  times  to  passive  failures, 
these  larger  attitudes  and  drifts  did  not  result  in  larger  heading  errors 
or  larger  lateral  displacements.  The  softover  failures  appeared  to  be  easy 
to  detect  and  act  upon. 
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Summary  of  pilot  performar.ee  during  control  failures.  Table  R-21  shows 
that  where  there  were  differences  in  pilot /system  performance  as  a  func¬ 
tion  or  monitor  mode,  the  addition  of  a  monitor  with  only  a  caution  light 
did  not  change  performance  fran  the  no-monitor  case.  During  passive 
failures,  the  aircraft  displacement  parameters,  altitude  used,  longitudinal 
range  consumed,  and  lateral  displacement  accrued  were  all  considerably 
reduced  by  the  monitor  with  caution  light  and  annunciator  panel.  With  the 
full  monitor  system  in  operation,  there  were  no  differences  between  soft- 
over  failures  and  passive  failures. 

Pitch  attitude,  vertical  velocity,  roll  attitude,  heading,  and  lateral 
drift  parameters  were  unaffected  by  the  monitor  modes  during  passive 
failures.  When  these  parameters  were  examined  across  both  passive  and 
softover  failures,  the  monitor  mode  effect  noted  in  the  displacement  para¬ 
meters  emerged. 
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Amount  of  change  required  far  pilot  fault  detection.  When  slightly 
rearranged,  these  data  can  provide  some  information  about  the  aTiount  of 
instrument  display  change  that  takes  place  before  a  pilot  detects  a  failure 
in  the  system  and  takes  action  on  that  failure  (without  a  monitor  to  help 
him).  Since  pilot  monitoring  capabilities  are  in  question,  the  data  shown 
in  the  next  two  tables  (Table  R-2 2  and  R-23)  were  gathered  with  special 
rules.  Where  a  reliable  difference  in  the  data  was  Indicated,  the  values 
were  taker  from  pilot  responses  during  no-monitor  approaches  for  failures 
in  the  axis  of  concern  and  of  the  indicated  failure  intensity.  Where  moni¬ 
tor  modes  were  not  significantly  different,  the  data  were  taken  from  means 
across  monitor  modes  for  the  failure  intensity  indicated  and  failures  in 
the  axis  of  concern. 

Of  the  pitch  axis  measures  taken.  Table  R-22  indicates  that  when  the 
pilots  detected  pitch  failures,  attitude  had  changed  0.6  degrees  and  verti¬ 
cal  velocity  had  changed  62.3  feet  per  minute.  These  small  values  suggest 
that  the  pilots  were  deriving  their  cues  from  other  sources  of  information; 
0.6  degrees  of  attitude  change  and  62  feet  per  minute  vertical  velocity 
change  over  a  7  to-15-second  time  do  not  seem  to  be  enough  of  an  average 
discrepancy  to  alert  the  pilot. 


TABLE  R-22 

Pitch  Axis  Conditions  When  Pilot  Responded  to  a  Failure 


Measures 

Failure  Intensity 

Attitude 

Difference 

Vertical 

Velocity 

Change 

Response 

Time 

Passive 

0.60  deg.  * 

(0.73  deg.)** 

62.3  fpm. 
(181.3  fpm.) 

15.68  sec. 
(3.3l)sec.) 

Soft over 

0.63  deg. 

(1.37  deg.) 

62.3  fpm. 
(181.3  fpm.) 

7.15  sec. 
(2.08  sec.) 

*  Means,  see  text  for  explanation  of  how  these  data  were  selected 

**  One  standard  deviation. 


Roll  Axis  measures  shown  in  Table  R-23  did  indicate  more  differences  than 
pitch.  Otoe  lateral  displacement  and  heading  error  build-up  seemed  to  trigger 
pilot  response  during  the  passive  roll  control  failures.  Roll  attitude  and  rate 
of  departure  from  the  localizer  seemed  to  trigger  pilot  responses  to  soft over 
failures.  Although  the  roll  attitudes  and  lateral  drifts  were  larger  during 
soft over  failures,  heading  error  had  not  yet  accrued  and  the  resulting  lateral 
displacement  was  lower  during  soft over  failures  at  pilot  response. 


TABUS  R-23 

Roll  Axis  Conditions  When  Pilot  Responded  to  a  Failure 


Measures 

Failure 

Intensity 

Attitude 

Difference 

Lateral 

Drift  Rate  Displacement 
Difference  Difference 

Heading 

Error 

Difference 

Response 

Time 

Passive 

3.1*  deg.* 
(2.6l  deg.)** 

6.74  fps .  29.94  ft. 

(5.73  fps.)  (28.45  ft.) 

0.74  deg. 
(1.37  deg.) 

15.68  sec. 
(3.31  »ec.) 

Softover 

8.80  deg. 
(2.77  deg.) 

11.04  fps.  15.10  ft. 
(5.55  fps.)  (17.35  ft.) 

0.18  deg. 
(0.96  deg.) 

7.15  sec. 
(2.08  sec.) 

*  Means,  see  text  for  explanation  of  how  these  data  were  selected. 
**  One  standard  deviation. 


In  the  Questionnaire,  the  pilots  were  asked  to  indicate  the  first  cue  that 
they  used  to  detect  a  control  system  failure  when  there  was  no  monitor  opera¬ 
ting.  Forty  percent  (40^)  of  the  pilots  indicated  glideslope  and  localizer 
deviation  and  2%  indicated  that  flight  director  commands  triggered  their 
decision.  Thirty-five  percent  (35%)  of  the  responses  were  varied;  "wandering 
off,"  "aircraft  flight  path  as  related  by  instruments,"  and  "failure  of  the 
aircraft  to  track  properly"  were  typical  responses  in  this  category.  The  data 
shown  in  Tables  R-2 2  aud  R-23  provide  some  measures  of  this  error  tolerance 
that  the  pilots  were  verbally  indicating. 
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The  difficulty  of  detecting  a  passive  failure.  The  difficulty  of  detec¬ 
ting  the  passive  failure  vas  indicated  by  pilot  responses  to  a  scale  in  the 
questionnaire.  The  median  responses  shown  in  Table  R-24  show  a  slight  but 
non-significant  difference  between  the  no-monitor  and  caution-light-only 
monitor  mode.  Pilots  indicated  that  it  vas  hard  to  detect  a  passive  failure 
in  these  modes.  With  the  caution  light  and  annunciator  panel  in  operation, 
pilots  reported  the  detection  of  passive  failures  io  be  easy. 


TABI£  R-24 

Passive  Failure  Detection  Difficulty 


Monitor 

Mode 

Difficulty  Scale  Categories 

Very 

Hard 

Slightly  Neither  Hard  Slightly 

Hard  Hard  Nor  Easy  Easy  Easy 

Very 

Easy 

No  Monitor 

X* 

Caution 

X 

Caution  + 

Annunciator 

X 

*  Median  response. 
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TABLE  R-25 


Concluding  remarks  on  control  failures.  Pilot  detection,  incorrect 
response,  false  alarm  scores  together  with  the  performance  data  and  opinion 
data  all  repeatedly  indicate  the  same  story — the  detection  of  passive  auto* 
pilot  control  failure  is  a  problem  for  the  pilot. 

With  no  monitor  system  operating,  it  seems  unlikely  that  any  single  air¬ 
craft  performance  parameter  would  provide  enough  information  for  the  pilot 
to  detect  a  failure.  If  a  single  parameter  would  have  provided  the  right 
information,  certakily  the  pilots  all  would  have  indicated  it  in  their 
questionnaire  responses,  and  that  parameter  would  have  "stood  out"  in  the 
performance  measurement  at  pilot  response.  The  data  support  a  common  sense 
notion  that  it  must  be  a  combination  of  events  that  would  lead  to  the 
detection  of  error  in  this  circumstance. 

In  the  aircraft  cockpit  it  is  probably  a  complex  interaction  of  the  amount 
of  error  in  several  different  parameters  combined  with  the  rate -of -error 
accrual  in  each  parameter  that  leads  to  pilot  detection  of  a  failure. 
Undoubtedly,  the  particular  parameters  that  might  be  used  are  different  from 
pilot  to  pilot.  Unfortunately,  the  problem  gets  even  more  complicated. 

As  a  monitor,  the  pilot  must  detect  the  presence  of  an  error  signal  in 
noise  that  is  created  by  the  environment,  turbulence  and  normal  autopilot 
corrective  control  actions.  The  reality  of  this  problem  was  indicated  by  one 
pilot  when  he  wrote,  "The  failure  had  to  show  enough  indication  to  assure 
that  it  was  not  just  an  error  in  synchronization  of  F/D  and  autopilot  since 
they  may  not  compensate  at  the  sane  rate."  An  error  in  synchronization  of 
the  flight  director  or  other  performance  and  attitude  instruments  may  be 
caused  by  either  an  abnormal  control  action  or  no  control  action  where  one 
should  be  required.  Pilots  have  to  look  for  the  former,  abnormal  control 
action  during  softover  control  failures,  and  the  latter,  no  control  action 
during  passive  failures.  The  difference  in  reported  difficulty  shows  that 
the  detection  of  no  control  action  where  one  is  required  is  much  more  diffi¬ 
cult  a  problem  than  the  detection  of  an  abnormal  control  action.  The  tone  of 
the  remarks  made  by  all  subject  pilots  was  suimnarized  nicely  by  one  subject 
when  he  indicated  that  the  no-monitor  condition  was  "...  a  very  dangerous 
situation. " 

When  the  caution  light  operated,  there  was  no  meaningful  improvement  in 
detection,  incorrect  respond,  false  alarms,  the  10  performance  measures, 
and  in  the  difficulty  scale.  It  was  felt  that  there  were  two  reasons  for 
this  lack  of  improvement.  First,  when  the  caution  light  tumed-on,  the  pilots 
still  had  to  find  the  problem;  in  effect,  the  light  simply  told  them  to 
"worry."  They  had  to  try  to  differentiate  between  a  control  system  failure 
and  a  display  system  failure.  Locating  the  problem  took  time.  Second,  it 
was  felt  that  the  mode  progress/fault  warning  system  that  was  in  the  simula¬ 
tor  did  not  present  an  optimum  environment  for  the  caution  light.  Normal 
mode  progress  lights  were  turning  to  indicate  the  "normal"  events  along 
the  bottom  of  the  mode  progress/fault  warning  display.  The  caution  light 
turning  on  would  indicate  an  "abnormal"  event.  Such  information  should  be 
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separated  on  the  panel  so  that  if  a  light  comes  on  in  the  fault  warning  loca¬ 
tion,  it  definitely  and  unequivocally  means  trouble. 

Finally,  performance  on  the  caution  light  plus  annunciator  panel  approaches 
•hawed  a  dramatic  improvement.  There  were,  again,  two  reasons  for  this  improve¬ 
ment.  First,  the  annunciator  panel  in  this  case  was  four  trana - illuminated 
switchlights  which  illuminated  red  in  the  appropriate  axis  when  a  failure 
occurred.  At  the  right-hand  aide  of  the  Captain's  instrument  display  panel, 
the  presence  of  a  red  light  meant  trouble  in  the  indicated  axis.  Secondly, 
and  mare  importantly,  the  pilot  corrective  action  was  to  push  the  svitchlight 
to  disengage  the  failed  axis.  The  red  light  served  as  a  command,  "push  me." 

Ho  pilots  questioned  this  command;  most  pilots  reacted  quite  favorably  to  it. 
This  test,  therefore,  suggests  that  fault  warning  devices  should  tell  the 
pilot  what  to  do  in  addition  to  what  is  wrong.  If  the  pilot  has  time,  the 
fault  warning  system  should  allow  the  pilot  to  diagnose  the  problem.  More 
is  said  about  the  characteristics  of  fault  warning  systems  in  the  last  sub¬ 
section  of  the  results  section. 


Display  Failures 

Display  failures  occurred  either  (l)  when  the  autopilot  was  fully  engaged 
and  there  had  been  no  previous  failure,  or  (2)  when  a  previous  control  system 
failure  had  caused  the  pilot  to  disconnect  an  axis  and  manually  control  the 
failed  axis.  Display  failures,  therefore,  occurred  during  three  different 
piloting  tasks:  (l)  while  flying  pitch  manually  with  the  roll  autopilot 
engaged,  (2)  while  flying  roll  manually  with  the  pitch  autopilot  engaged,  and 
(3)  while  monitoring  the  fully  engaged  autopilot.  Pilot  detection  of  display 
failures  was  quite  dependent  upon  the  piloting  task  for  each  of  the  principle 
display  variables  (failure  mode  and  failure  location). 

The  effect  of  monitor  mode  and  control  task  upon  detection.  Table  R-26 
shows  that  the  number  of  times  that  pilots  did  not  detect  display  failures 
decreased  as  the  caution  light  was  added,  then  further  decreased  as  the  full 
monitor  system  was  in  operation.  Out  of  60  times  that  display  failures  were 
undetected  (out  of  648  approaches),  nine  failures  were  missed  with  the  full 
monitor  system  in  operation.  Only  two  failures  were  undetected  when  the 
autopilot  was  fully  operating  and  the  pilots  were  required  only  to  monitor 
automatic  system  performance. 

Failure  detection  during  split-axis  control  did  not  fare  as  well.  Thirty- 
four  (34)  display  failures  went  undetected  when  the  pilots  were  manually  con¬ 
trolling  pitch.  The  worst  condition  (l8  no-detections)  occurred  during  roll 
mamifli  approaches  with  no  monitor  in  operation. 
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TABIZ  R-26 

No  Detection,  Incorrect  Response  and  Incorrect  Decision 
to  Land  During  Display  Failures 


Measure 

Pilot 

Control  Task 

No 

Monitor 

Monitor  Mode 

With  Caution 

Caution  Annunciator 

Totals 

No 

Pitch 

11* 

10 

3 

24 

Detection 

Manual 

15-3% 

13.9% 

4.2% 

11.1% 

Roll 

18 

10 

6 

34 

Manual 

25.0% 

13.9% 

8.3% 

15.7% 

Automatic 

1 

1 

0 

2 

1.4% 

1.4% 

0.0% 

0.9% 

Totals 

30 

21 

9 

6o 

13.9% 

9.7% 

4.2% 

9.3% 

Incorrect 

Pitch 

18 

14 

14 

46 

Response 

Manual 

25.0 % 

19-4% 

19.4% 

21.3% 

Roll 

27 

16 

10 

53 

Manual 

37.5% 

22.2% 

13.9% 

24.5% 

Automatic 

10 

12 

5 

27 

13-9% 

16.7% 

6.9% 

12.5% 

Totals 

55 

42 

29 

126 

25.5% 

19.4% 

13.4% 

19.4% 

Incorrect  Pitch 

13 

10 

11 

34 

Decision  Manual 

to  Land 

18.1% 

13.9% 

15.3% 

15.7% 

Roll 

19 

11 

6 

36 

Manual 

26.4% 

15.3% 

8.3% 

16.7% 

Automatic** 

7 

4 

2 

13 

9.7% 

5.6% 

2.8% 

6.0% 

Totals 

39 

25 

19 

83 

18.1% 

11.6% 

QM 

12.8% 

*  Number  of  cases.  Percent  shown  is  percent  of  occurrance  of  no  detection, 
incorrect  response,  or  incorrect  decisions  to  land  out  of  the  total  number 
of  display  failures  given  during  the  indicated  pilot  task  and  monitor  mode. 

**  In  these  cases,  the  pilots  had  not  detected  a  previous  passive  control 
failure  and  thought  that  they  were  "automatic."  Since  they  were  operating 
as  system  monitors,  their  "incorrect  decisions  to  land"  were  scored  in 
this  category. 
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Incorrect  response s.  An  incorrect  response  was  scored  when  a  pilot  did  not 
correctly  identify  the  display  failure.  Contrary  to  the  incorrect  response 
performance  during  control  system  failures,  the  number  of  incorrect  responses 
reduced  when  the  caution  light  was  added.  The  55  incorrect  responses  during 
no-monitor  approaches  reduced  to  k2  with  the  addition  of  a  caution  light. 

When  the  annunciator  panel  operated,  the  number  of  incorrect  responses  further 
reduced  to  29,  Again,  roll  manual  produced  more  incorrect  responses  (53) 
overall,  and  the  roll -manual,  no-monitor  situation  was  the  worst  case  with  27 
incorrect  responses.  Although  only  two  failures  were  undetected  when  the  pilots 
were  monitoring  the  operation  of  the  automatic  system,  the  pilots  made  27 
incorrect  responses  to  display  failures  while  automatic.  Overall,  the  severity 
of  the  experimental  fault  detection  task  was  reflected  by  a  20£  error  rate. 

As  a  consequence  of  this  high  error  rate,  over  two-thirds  of  the  126  incorrect 
responses  led  to  a  touchdown  when  the  pilots  should  have  gone  around. 

Incorrect  decision  to  land.  Eighty-three  (83)  times  the  pilots  landed  when 
the  information  upon  which  they  were  depending  to  manually  fly  the  simulator 
had  failed.  Table  R-26  shows  that  roll  manual,  no-monitoj;  was  again  the  worst 
case.  With  the  full  monitor  system  in  operation,  however,  pitch  manual 
resulted  in  more  incorrect  decisions  than  roll  manual  control.  The  full  moni¬ 
tor  system  reduced  the  number  of  times  that  incorrect  decisions  to  continue 
were  made  to  19  out  of  216,  or  about  9/6  of  the  time. 

The  incorrect  decisions  to  land  when  the  pilot  was  "automatic"  needs  further 
explanation.  In  no  case  was  it  necessary  for  the  pilot  go -around  because  of  a 
display  failure  when  the  full  autopilot  was  operational.  For  example,  a 
Captain's  glide si ope  failure  did  not  affect  the  autopilot  and  did  not  require 
a  go-around  under  the  decision  rules  set  for  this  experiment.  There  were,  how¬ 
ever,  13  times  that  the  pilots  had  not  detected  a  previous  passive  control 
system  failure  by  the  time  that  the  second  failure  (the  display  failure- 
inserted  no  sooner  than  20  seconds  after  the  insertion  of  the  previous  control 
failure)  was  inserted  in  the  same  axis  as  the  control  failure.  Since  the 
pilots  were  not  in  manual  control  (even  though  they  should  have  been),  they 
were  operating  as  monitors  of  what  they  thought  was  autopilot  performance. 

These  13  incorrect  decisions  to  land  were  placed  in  the  'Automatic"  pilot  con¬ 
trol  task  category  in  Tables  R-26  and  R-34. 

Typically,  two  types  of  responses  occurred  during  these  13  incorrect 
decislmsto  land  while  the  pilots  were  "automatic."  The  pilots  either 

(1)  responded  correctly  to  the  display  failure  but  failed  to  realize  that  the 
autopilot  was  not  fully  operational  and  continued  to  "automatic"  landing,  or 

(2)  they  responded  to  the  previous  passive  control  failure  after  the  display 
failure  had  occurred  and  ignored  the  display  failure.  Since  they  had  made  a 
response  to  the  second  failure,  they  were  apparently  satisfied,  and  continued 
manually  in  the  failed  axis  to  landing  with  the  flag  in  full  view.  Putting 
this  second  type  of  response  in  different  terms,  two  failures  occurred.  The 
pilots  did  not  see  the  first  failure.  When  the  second  failure  occurred,  they 
responded  to  the  first  and  did  not  take  further  action  cm  the  second  failure. 
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Tables  R-27  through  R-33  present  summaries  of  touchdown  parameters  during 
75  of  the  83  accidental  landings.  The  data  of  eight  accidental  landings 
were  lost  because  the  touchdowns  occurred  too  rapidly  for  the  measurement 
system  control  logic  in  the  simulator  to  adequately  respond.  The  tables  show 
the  data  according  to  the  actual  conditions  (pitch  manual,  roll  manual)  that 
existed,  independent  of  whether  or  not  the  pilot  knew  that  the  control  sys¬ 
tem  failure  had  occurred.  The  principle  result  was  that  lateral  plane  para¬ 
meters  indicated  some  unsuccessful  landings  during  roll  axis  failure  and 
vertical  plane  parameters  showed  some  unsuccessful  landings  during  pitch 
failures.  As  will  be  seen,  some  of  these  accidental  landings  would  have 
been  successful  by  criteria  applied  to  the  analysis  of  normal  touchdowns. 
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of  landings  when  score  was  within  the  defined  intervals 


TABLE  R-30 

Lateral  Displacement  at  Accidental  Touchdown 


of  landings  when  score  was  in  the  defined  intervals 


TABLE  R-32 

Lateral  Drift  at  Accidental  Touchdown 


TABLE  R-33 

Heading  Error  at  Accidental  Touchdown 
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of  landings  when  the  score  was  within  the  defined  interval. 


Other  incorrect  responses.  Thirty- three  of  the  remaining  forty- three 
incorrect  responses  had  two  principle  causes.  First,  there  were  19  times  that 
the  pilots  disengaged  the  autopilot  completely  at  the  sight  of  the  second 
failure  when  it  was  unnecessary  to  do  so.  Second,  during  an  additional  14 
of  the  incorrect  responses,  the  pilots  conservatively  elected  to  go-around 
when  they  could  have  continued.  In  a  very  real  sense,  this  was  hardly  an 
error,  but  it  was  an  incorrect  response  according  to  the  decision  rule  set 
for  this  study. 

Sun MTV  nf  the  effect  of  monitor  mode  and  control  task  upcp  detection. 

With  the  full  monitor  system,  therefore,  nine  display  failures  were  undetec¬ 
ted  during  split-axes  approaches.  All  display  failures  were  detected  when 
the  pilots  were,  or  thought  that  they  were,  automatic.  Detection  performance 
was  spoiled  by  the  number  of  incorrect  responses.  During  pitch  manual,  con¬ 
trol  with  the  full  monitor  system  in  operation,  14  incorrect  responses  resulted 
in  11  incorrect  decisions  to  land.  During  roll  manual  control,  10  incorrect 
responses  resulted  in  six  incorrect  landings.  This  resulting  pilot/system 
performance  with  the  full  monitor  system  in  operation  was  not  felt  to  be  good 
enough,  even  though  the  test  was  quite  severe. 

The  effect  of  display  failure  «n*  control  task  upon  detection. 

Flag  failures  occurred  in  three  locations:  (l)  in  the  ADI,  (2)  in  the  HSI,  and 
(3)  in  the  radar  altimeter  (BA)  instrument  immediately  to  the  right  of  the  ADI. 
A  fourth  display  failure,  vertical  gyro  unreliable,  was  presented  to  examine 
an  insidious  failure  that  was  not  associated  with  flag  events.  Table  R-34 
shows  pilot  detection  performance  as  a  function  of  display  failure  location 
and  pilot  control  task. 

Ho  detections.  Table  R-34  shows  that  the  two  no-detections  during  automa¬ 
tic  control  occurred  during  an  HSI  flag  and  a  RA  flag  failure.  When  the 
pilots  were  roll  manual,  they  missed  fewer  -HSI  flags  than  all  other  flags. 

When  pitch  manual,  the  pilots  missed  more  HSI  flags.  The  data  seem  to  suggest 
that  the  pilots  pay  more  attention  to  the  radar  altimeter  when  pitch  manual 
(two  no-detections)  than  when  roll  manual  (10  no-detections).  The  vertical 
gyro  unreliable  failure  was  detected  more  frequently  when  pitch  manual.  Quite 
likely,  the  pilots  adopted  different  scan  patterns  for  the  different  control 
tasks.  Over  all  control  tasks,  the  ADI  and  HSI  flags  were  missed  more  fre¬ 
quently  (17  each)  than  the  radar  altimeter  flag  (13)  and  the  vertical  gyro 
unreliable  (13). 


66 


TABI£  R-34 

No  Detection,  Incorrect  Response  and  Incorrect  Decision 
to  Land  During  Display  Failures 


Measure 

Pilot 

Control  Task 

Failure  Location 

ADI  HSI  RA 

Flag  Flag  Flag 

VG 

Unreliable 

Total 

No 

Pitch 

7* 

10 

2 

5 

24 

Detection 

Manual 

13.03 6 

t— 1 

00 

* 

3.736 

9.336 

11.136 

Roll 

10 

6 

10 

8 

34 

Manual 

18.5% 

11.136 

18.536 

14.836 

15.736 

Automatic 

0 

1 

1 

0 

2 

0.036 

1.936 

1.936 

0 .036 

0.916 

Total 

17 

17 

13 

13 

60 

10.536 

10.536 

8.036 

8.036 

9.336 

Incorrect 

Pitch 

13 

8 

16 

9 

46 

Response 

Manual 

24.036 

14.836 

29.736 

16.736 

21.336 

Roll 

8 

18 

12 

15 

53 

Manual 

14.836 

33.336 

22.236 

27.836 

24.536 

Automatic 

14 

3 

3 

7 

27 

25.9% 

5.6$ 

5.636 

13.036 

12.536 

Total 

35 

29 

31 

31 

126 

21.636 

17.936 

19.136 

19.136 

19.436 

Incorrect 

Pitch 

12 

0 

15 

7 

34 

Decision 

Manual 

22.236 

0.036 

27.836 

13.036 

15.736 

to  Land 

Roll 

5 

17 

0 

l4 

36 

Manual 

9.336 

31.536 

0.036 

25.936 

16.736 

Automatic** 

9 

1 

1 

2 

13 

16.836 

1.936 

-.936 

3.736 

6.036 

Total 

26 

18 

16 

23 

83 

16.136 

11.136 

9.936 

14.236 

12.916 

*  Number  of  cases.  Percent  shown  is  percent  of  occurrence  of  no  detection, 
incorrect  response,  or  incorrect  decision  to  land  out  of  the  total 
number  of  display  failures  given  during  the  indicated  pilot  task  end 
failure  location. 

#*  See  footnote  on  Table  R-26. 
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Incorrect  responses.  Curiously,  18  of  the  HSI  flags  during  roll  laanual 
situations  led  to  Incorrect  responses  and  13  of  the  pitch  manual  ADI  flags 
led  to  incorrect  responses .  Note  that  the  HSI  flags  during  roll  manual  and 
the  ADI  flags  during  pitch  manual  were  the  best  detection  conditions  during 
split-axis  control;  these  same  conditions  were  the  worst  in  terms  of  incor¬ 
rect  responses.  Fewer  ADI  flags  led  to  incorrect  responses  during  roll  manual 
control  (8)  than  during  pitch  manual  control  (13)*  Fewer  HSI  flags 
led  to  incorrect  responses  during  pitch  manual  control  (8)  than  during  roll 
manual  control  (l8).  Only  two  HA  flags  were  missed  during  pitch  ip»nn»i 
control,  yet  there  were  incorrect  responses  to  l6  of  the  RA  failures 
during  pitch  manual  control.  All  ADI  flags  were  detected  during  automatic 
control;  yet,  l4  responses  to  ADI  flags  were  incorrect  while  the  pilot  was 
monitoring  the  autopilot.  Most  incorrect  responses  were  incorrect  identi¬ 
fications  of  the  flags. 

These  data  reflect  that  it  is  not  enough  to  just  see  the  problem;  the  pilot 
must  be  able  to  correctly  respond  to  what  he  sees.  These  data  are  a  clear 
case  far  the  necessity  of  labeling  instrument  flags  and  having  the  flags  cover 
the  functions  that  are  failed. 

Incorrect  decisions  to  land.  During  pitch  manual  control,  all  ADI  flags, 

RA  flags  and  vertical  gyro  failures  required  a  go- around.  For  these 
failures,  almost  all  Incorrect  responses  resulted  in  an  Incorrect  decision 
to  land.  While  pitch  manual  (roll  auto)^  an  HSI  flag  was  not  a  go -around  situ¬ 
ation  an&  therefore,  resulted  in  no  incorrect  decisions  to  land.  The  same  was 
true  of  the  radar  flag  during  roll  manual  approaches.  During  roll  manual,  all 
HSI  flags,  vertical  gyro  unreliable  and  the  ADI  vertical  gyro  flagp required  a 
go-around.  Incorrect  responses  to  these  failures  almost  always  led  to  an 
incorrect  decision  to  land. 

Summary.  During  pitch  manual  control,  most  non-detections  occurred  on  the 
HSI  although  seven  ADI  flags  were  missed.  During roll  manual  control,  most 
non-detections  occurred  on  the  ADI  and  radar  altimeter  (10  each).  Eight  (8) 
vertical  gyro  unreliable  failures  were  missed  and  six  (6)  HSI  flags  were 
undetected.  ADI  and  HSI  flags  were  missed  more  than  radar  flags  and  the  verti¬ 
cal  gyro  unreliable  failures.  Most  incorrect  responses  occurred  to  ADI  flags 
(35)  although  HSI  (29),  radar  (3l)»  and  vertical  gyro  unreliable  failures  (3l) 
were  not  far  behind.  The  largest  number  of  incorrect  responses  (l8)  occurred 
to  HSI  flags  during  roll  manual  control.  Curiously,  as  the  probability  of 
detection  increased,  the  number  of  incorrect  responses  also  increased  for  a 
particular  control  task. 


*A  vertical  gyro  failure  while  split- axes  required  the  pilot  to  relin¬ 
quish  control  to  the  First  Officer  in  this  experiment.  This  failure  was, 
therefore*  not  well  received  by  the  pilots,  who  confessed  operational  reluc¬ 
tance  to  transfer  control  at  the  critical  moment  of  go-around  during  these  low 
altitude  failures. 
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As  previously  noted,  a  pilot's  ability  to  detect  failures  and  respond 
correctly  is  seriously  degraded  when  he  is  actively  in  control  of  one  or 
more  axes.  When  he  is  only  monitoring  a  fully  automatic  system,  his  ability 
to  detect  failures  is  good,  but  his  ability  to  respond  correctly  is  not  as 
good  as  desired.  Perhaps  the  responses  seen  in  this  study  would  be  better 
with  more  extensive  training. 

Having  shown  pilot  detection,  non-detection  and  incorrect  responses  during 
display  failures,  pilot/system  performance  changes  during  display  failures 
will  next  be  examined.  Pilot  response  time,  pitch  attitude,  vertical 
velocity,  altitude  used,  longitudinal  range  consumed,  indicated  airspeed 
change,  lateral  displacement,  lateral  drift,  heading  error  and,  finally, 
roll  attitude  comprise  the  system  performance  measurement  set. 

Pilot  response  time.  Response  time  indicated  the  amount  of  time  con¬ 
sumed  between  display  failure  insertion  and  pilot  response  to  that  failure. 
Table  R-35  shows  that  pilot  responses  to  ADI  flags  were  faster  (5.56  sec.) 
than  responses  to  HSI  and  radar  altimeter  flags  (RA)  and  the  vertical  gyro 
unreliable  (VGU)  failure  (average  7.36  sec.  to  HSI,  RA,  and  VGU).  The  fail¬ 
ure  monitor  mode  did  not  influence  pilot  response  time  to  display  failures. 
Neither  was  pilot  response  time  affected  by  pilot  control  task  (split-axes 
or  automatic). 


TABLE  R-35 

Pilot  Response  Time  During  Display  Failures 


Display  Failures 


Measure 

ADI 

HSI 

RA 

VGU 

Mean 

5.56*  | 

7.30 

7.64 

7.15 

Standard 

3.46 

4.79 

5.03 

5.06 

Deviation 


*  Seconds# 


Pitch  attitude.  The  change  in  pitch  attitude  between  failure  insertion 
and  pilot  response  was  high  (pitch-up)  and  more  variable  (l)  during  pitch 
manual  control  when  the  radar  altimeter  had  failed  and  (2)  during  roll 
mnmial  control  when  the  vertical  gyro  unreliable  failure-  occurred 
(Table  R-3o).  The  pitch  manual  radar  altimeter  failures  and  roll  manual 
vertical  gyro  failures  were  go-around  conditions.  These  data  suggest  that 
pilots  had  started  to  execute  go-around  prior  to  triggering  the  response 
measurement  by  engaging  the  flight  director  go-araund  mode  or  disengaging 
the  autopilot  during  these  conditions.  No  such  pitch-up  action  was  evi¬ 
denced  during  other  go-around  situations  (£DI  and  VGU  failures  during  pitch 
manual  and  HSI  failures  during  roll  manual,/.  The  apparent  initiation  of 
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go-around  prior  to  triggering  response  was  also  seen  in  the  vertical  velocity 
data. 


TABLE  R-36 

Pitch  Attitude  Change  During  Display  Failures 


Pilot 

Control 

Task 

Display  Failures 

Across 

Display 

Failure 

ADI 

HSI 

RA 

VGU 

Pitch 

-0.45  * 

-0.23 

-2.20** 

-0.63 

-0.88 

Manual 

(2.12)*** 

(1.56) 

(4.57) 

(2.41) 

(2.98) 

Roll 

0.00 

-0.63 

-0.06 

-1.46** 

-0.51 

Manual 

(0.62) 

(2.49) 

(0.93) 

(4.31) 

(2.61) 

Automatic 

0.20 

(0.99) 

-0.11 

(0.72) 

0.03 

(0.39) 

0.24 

(0.24) 

0.09 

(0.75) 

Across 

Control 

-0.08 

-0.32 

-0.71 

-0.6l 

-0.43 

Task 

(1.41) 

(1.75) 

(2.89) 

(2.95) 

(2.36) 

*  Mean  difference  in  degrees  between  failure  insertion  and  response. 
**  These  two  scores  significantly  different  from  all  the  rest,  but 
not  from  each  other.  Remaining  scores  are  not  significantly 
different. 

***  One  standard  deviation. 


Vertical  velocity.  Vertical  velocity  during  display  failures  changed  as 
a  complex  interaction  of  (l)  monitor  mode,  (2)  the  particular  display  failure, 
(3)  pilot  control  task,  and  (4)  pilot  group  differences.  These  data  are 
presented  by  failure  monitor  mode,  starting  with  no  monitor  approaches. 

No  monitor.  During  no  monitor  approaches,  the  two  pilot  groups  had 
exactly  the  same  conditions.  Vertical  velocity  decreased  during  the  pitch 
manual,  radar  altimeter  failure  for  both  groups  (Table  R-37).  The  decrease 


^”As  indicated  in  the  method  section,  the  pilots  were  placed  into  two 
groups;  one  group  had  a  "Flag"  light  for  flag  failures  during  the  "caution" 
monitor  mode  and  the  other  group  had  the  caution  light.  During  "no  monitor" 
conditions  both  pilot  groups  had  equivalent  conditions.  The  analysis  of 
variance  results,  however,  indicated  group  differences  during  the  no  monitor 
condition.  These  two  pilot  groups  were  not  equivalent  during  equivalent 
conditions;  therefore,  the  flag  light  variable  was  confounded  with  pilot 
group  differences. 
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in  vertical  velocity  was  evidence  that  a  go-around  had  already  been  initiated 
when  the  pilots  responded  by  disengaging  the  autopilot  or  engaging  the  go- 
around  flight  director  mode.  Group  differences  were  apparent  during  the 
vertical  gyro  unreliable  failure.  Group  II  roll  manual  had  little  vertical 
velocity  change;  yet,  Group  I  pilots  during  the  same  conditions  had  initia¬ 
ted  a  go-around.  When  the  monitor  "with  caution  light"  was  added,  vertical 
velocity  change  was  quite  different. 

TABLE  R-37 


Vertical  Velocity  Change  During  No  Monitor  Display  Failures 


Group 

Manual 

Axis 

Display  Failure 

ADI  HSI  RA 

VGU 

Across 

Display 

Failures 

I 

Pitch 

-54* 

-157 

-394 

-39 

-l6l 

(309)** 

(363) 

(982) 

(287) 

(558) 

Roll 

98 

-85 

-148 

-319 

-113 

(225) 

(214) 

(250) 

(914) 

(500) 

II 

Pitch 

-169 

-21 

-512 

-258 

-24o 

(290) 

(351) 

(1045) 

(529) 

(628) 

Roll 

60 

-129 

-143 

-56 

-671 

(202) 

(431) 

(192) 

(182) 

(273) 

*  Mean  difference  in  feet  per  minute  between  failure 


insertion  and  response. 
**  One  standard  deviation. 


With  caution.  This  monitor  mode  has  been  called  "with  caution"  for  brevi¬ 
ty.  Actually,  this  mode  contained  the  caution  light,  flag  light  and  deviation 
light  variables.  Group  I  had  a  caution  and  deviation  light  operating.  Group 
II  had  a  flag  light  for  flag  failures  or  a  caution  light  for  vertical  gyro 
unreliable  failures.  Table  R-38  shows  what  happened. 

With  caution  and  deviation  light  (Group  I)  operating,  go-around  appeared 
in  progress  during  pitch  manual  ADI  flags  and  during  roll  manual  vertical 
gyro  unreliable  failures.  Whereas  Group  I  pilots  were  executing  go-arounds 
to  radar  altimeter  flags  during  pitch  manual  with  no  monitor  (Table  R-37J> 
they  stopped  doing  this  when  the  caution  and  deviation  light  operated  (Table 
R-38).  Group  II  pilots  further  decreased  their  vertical  velocity  during 
radar  altimeter  failures  while  pitch  manual.  Since  the  nominal  vertical 
velocity  during  the  approach  was  -640  feet  per  minute,  the  -704  feet  per 
minute  average  value  of  vertical  velocity  change  (difference  between  failure 
insertion  and  pilot  response)  indicated  that  a  positive  climb  rate  was 


established  prior  to  engaging  the  go-arovnd  mode  or  disengaging  the  roll 
autopilot.  Pilots  in  Group  II  were  responding  to  the  roll  manual  HSI  flags 
in  the  same  way.  Further  comment  will  be  deferred  until  the  final  part  o ' 
this  interaction,  the  caution  plus  annunciator  monitor  mode,  has  been 
presented. 


TABLE  R-38 

Vertical  Velocity  Change  During  "Caution"  Mode  Display  Failures 


Display  Failure 

Across 

Group 

Manual 

Display 

Axis 

ADI 

HSI 

RA 

VGU 

Failures 

Pitch 

-415* 

71 

-173 

162 

-87 

Caution 

I:  + 

(997)**  (271) 

(679) 

(301) 

(649) 

Deviation 

Roll 

-26 

-46 

19 

-372 

-106 

(174) 

(241) 

(206) 

(975) 

(522) 

Pitch 

-300 

-268 

-704 

-212 

-307 

Flag 

(445) 

(589) 

(862) 

(465) 

(619) 

II:  or 

-56 

-65 

-163 

Caution 

Roll 

-520 

-30 

(218) 

(822) 

(319) 

(153) 

(486) 

*  Difference  in 

feet  per 

na.nute  between  failure 

insertion  and 

response. 

**  One  standard  deviation. 


Caution  plus  annunciator.  When  the  full  monitor  system  was  operating,  both 
groups  again  had  the  same  conditions.  More  evidence  of  executing  go-arounds 
prior  to  engaging  the  go-around  mode  was  seen.  Table  R-39  shows  both  groups 
executing  go-around  during  pitch  manual  radar  altimeter  flags  and  vertical 
gyro  unreliable  failures.  Whereas  Group  I  executed  go-arounds  in  this  fashioi; 
during  "caution  light,"  pitch  manual  ADI  failures  (see  Table  R-38),  their 
vertical  velocity  was  increasing  by  266  feet  per  minute  (sinking  more  rapidly) 
during  the  same  conditions  with  full  monitor  plus  annunciator  panel  in  opera¬ 
tion  (Table  R-39).  Group  I  showed  a  similar  reversal  of  strategy  during  ver¬ 
tical  gyro  unreliable  failures  while  roll  manual;  in  the  "caution  light"  mode 
(Table  R-38),  Group  I  pilots  had  started  go-around,  but  exhibited  less 
tendency  to  go-around  under  the  same  conditions  with  the  full  system  in 
operation  (Table  R-39)* 
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Group  II  strategy  also  changed.  During  roll  manual  IE  I  flags  with  a  flag 
light  (Table  R-38),  Group  II  was  cloarly  executing  go -around  prior  to  response 
(average  velocity  change  of  -520).  During  the  same  conditions  with  the  full 
monitor  system  (Table  R-39)  there  was  only  an  average  12  foot  per  minute 
change  in  vertical  velocity  at  response. 


TABLE  R-39 

Vertical  Velocity  Change  During  Display  Failures 
With  The  Full  Monitor 


Manual 

Display  Failure 

Across 

Display 

Failures 

Group 

Axis 

ADI 

HSI 

RA 

VGU 

Pitch 

266* 

-60 

-548 

-598 

-235 

I 

(542)**  (225) 

(1072) 

(H93) 

(893) 

Roll 

9 

(227) 

-402 

(902) 

-12 

(103) 

171 

(457) 

-59 

(542) 

II 

Pitch 

-515 

(589) 

38 

(412) 

-479 

(1096) 

-323 

(749) 

-320 

(755) 

Roll 

12 

-1SS1 

122 

(295) 

-6 

-513 

-iml 

-96 

(S3*) 

*  Mean  difference  in  feet  per  minute  between  failure 


insertion  and  response. 
**  One  standard  deviation. 


Discussion.  During  no-monitor  approaches  with  vertical  gyro  unreli¬ 
able  failures,  basic  group  differences  were  evident.  These  group  differ¬ 
ences  verified  our  subjective  judgment  that  the  pilot  groups  were  not 
equivalent.  One  group  seemed  to  have  more  difficulty  adopting  the  required 
decision  logic  and  executing  their  decisions  in  the  short  time  required.  Out 
of  all  the  performance  data  taken,  only  vertical  velocity  change  showed  group 
differences. 

The  flag  light  may  have  had  seme  value  when  the  annunciator  panel  was 
not  operating.  Seven  out  of  nine  pilots  who  saw  the  flag  light  (78 %)  report¬ 
ed  it  to  be  of  significant  value  in  their  questionnaires.  Vertical  velocity 
showed  more  go-arounds  in  progress  when  response  was  triggered  during  the 
flag  light  operation  than  during  caution  light  operation  for  the  same  fail¬ 
ures.  The  basic  group  differences,  however,  could  have  contributed  to  the 
performance  seen  during  flag  light  only  conditions,  and  thus  confounded  these 
data.  Since  overall  performance  was  more  consistent  during  the  full  monitor 
system  operation,  it  must  be  concluded  that  the  flag  light  may  have  had  some 
benefit,  but  the  full  system  with  the  annunciator  was  better. 
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At  one  time  or  another,  the  pilots  initiated  go-around  before  triggering 
"response"  measurement  by  engaging  the  go-around  flight  director  mode,  dis¬ 
engaging  the  autopilot  or  pushing  the  response  bar.  Their  strategy  seemed  to 
be  a  simple  matter  of  priority:  FIRST,  act  to  initiate  a  positive  climb  rate, 
THEN,  engage  go-around,  disconnect  the  autopilot  and  clean-up  the  aircraft  as 
time  permitted.  Unfortunately,  this  pilot  strategy  desensitized  the  response 
measures  taken  during  display  failures,  along  with  some  of  the  go-around 
measurement. 

"Response"  was  not  always  triggered  at  the  same  point  in  the  detection- 
decision-response  sequence  of  events.  Where  a  go-around  was  required,  more 
go-arounds  were  in  progress  at  response  during  the  full  caution-plus  annun¬ 
ciator  monitor  mole  than  during  the  with-caution  monitor  mode;  the  with- 
caution  mode  elicited  more  go-arounds  in  progress  at  "response"  than  during 
the  no-monitor  mode.  If  pilots  had  triggered  response  measurement  prior  to 
taking  their  go-around  action  in  all  cases,  then  the  response  time  measures 
may  have  shown  proportionally  shorter  response  times  to  the  monitor-with- 
caution  and  the  monitor -with- caution -and -annunciator  modes.  As  it  was, 
response  time  was  not  sensitive  to  any  monitor  mode  differences.  For  the 
same  reason,  all  display  failure  response  measurement  was  desensitized  dur¬ 
ing  the  caution  and  caution-plus-annunciator  modes.  These  data,  however,  do 
reflect  the  time  and  parameter  changes  from  failure  insertion  until  the  pilot 
had  the  first  opportunity  to  hit  the  response  bar,  disconnect  the  autopilot, 
or  engage  go-around,  which  is  certainly  valuable  information. 

Altitude  used.  An  average  62.95  feet  was  consumed  between  failure  inser¬ 
tion  and  pilot  response  during  all  display  failures.  The  standard  deviation 
shows  that  68%  of  the  time  altitude  consumed  was  within  +  53.33  feet  of  that 
mean.  Failure  monitor  modes,  pilot  control  task  and  failure  location  did 
not  systematically  affect  the  amount  of  altitude  used. 

Longitudinal  range.  The  differences  between  longitudinal  range  at  fail¬ 
ure  insertion  and  pilot  response  indicated  the  distance  traveled  while  the 
pilot  was  detecting  arid  acting  upon  the  failure.  Similar  to  response  time, 
monitor  mode  did  not  influence  distance  traveled,  but  display  failure  loca¬ 
tion  did.  Table  B-40  shows  that  the  average  distance  traveled  during  ADI 
failures  was  1,133  feet,  while  the  distance  traveled  during  all  other  display 
failures  was  about  1,515  feet  on  the  average.  As  the  range  increased,  so  did 
the  variability. 


TABLE  R-40 


Longitudinal  Range  Traveled  During  Display  Failures 


Measure 

ADI 

Display 

HSI 

Failure 

RA  VGU 

Mean 

1133* 

Ii4q6 

1584  1465 

1  Standard  Deviation 

727 

1004 

1031  1026 

*  Distance  in  feet. 

Dote:  Means  contained  within  the  box  are  not  significantly 
different. 
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Indicated  airspeed.  Airspeed  did  vary  slightly  as  a  function  of  failure 
monitor  mode  (Table  R-4l ) .  When  there  was  no  monitor  at  all,  airspeed 
increased  only  0.1  knots  on  the  average.  When  the  monitor  operated  in  both 
the  caution  light  and  annunciator  panel  modes,  airspeed  increased  an  average 
1.7  knots  between  failure  insertion  and  pilot  response.  Practically,  this 
small  average  change  in  airspeed  may  seem  meaningless;  but,  note  the  standard 
deviations. 


TABLE  R-4i 

Indicated  Airspeed  Change  During  Display  Failures 


Measure 


Monitor  Mode 

No  With  Caution  + 
Monitor  Caution  Annunciator 


Mean  -0.11*  I  -1.53  -T. 
1  Standard  Deviation  4.77  6.70  7, 
*  IAS  change  in  knots 

Note:  Means  contained  within  the  box  are  not 
significantly  different. 


In  the  two  modes  of  monitor  operation,  airspeed  change  at  pilot  response 
varied  around  the  indicated  means  +  approximately  7  knots  68%  of  the  time. 
Therefore,  32%  of  the  time  airspeed  deviated  in  excess  of  +  7  knots  about  the 
mean.  Since  the  vertical  velocity  and  pitch  aEtitu&e measures  revealed  that 
the  pilots  had  initiated  go-around  prior  to  triggering  the  response  measure¬ 
ment,  the  variability  of  airspeed  with  the  monitor  system  in  operation  also 
must  have  been  due  to  thrust  application  prior  to  triggering  response  during 
go-around  decisions. 

Lateral  displacement.  The  average  change  in  lateral  displacement  from 
display  failure  insertion  until  pilot  response  varied  as  a  function  of  pilot 
control  task,  display  failure  and  monitor  mode  during  roll  manual  approaches. 
Disregarding  monitor  mode  for  the  moment.  Table  R-42  indicates  that  average 
lateral  displacement  changed  most  during  radar  altimeter  failures  when  the 
pilots  were  roll  manual  (62.29  feet). 


The  next  largest  change  was  29.13  feet  during  HSI  failures  when  roll 
manual  and,  finally,  an  average  7.86  feet  during  HSI  failures  when  pitch  manu¬ 
al.  Note  that  the  variability  of  these  data  was  quite  high,  indicating  a 
great  deal  of  lateral  displacement  change  during  roll  manual  display  failures 

Table  R-43  shows  that  under  no  monitor,  roll  manual  conditions,  lateral 
displacement  changed  an  average  154  feet  when  radar  altimeter  failures  occur 
ed.  Especially  note  that  one  standard  deviation  during  this  condition  was 
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271  feet.  Sixteen  percent  (16$)  of  the  time,  lateral  displacement  changed 
more  than  271  plus  154,  or  425  feet. 


TABLE  R-42 

Lateral  Displacement  Change  During  Display  Failures*** 


Pilot 

Control 

Task 


Display  Failure 


ADI 


HSI 


RA 


VGU 


Across 

Display 

Failures 


-0.06* 

7.86 

0.22 

-5.99 

0.6l 

(19.74)** 

(49.15) 

(25.79) 

(46.88) 

(37.69) 

-10.94 

29.13 

62.29 

-6.17 

18.58 

(80..68) 

(i4o.n) 

(192.06) 

(143.50) 

(146.74) 

2.19 

-0.35 

1.42 

-5.59 

-0.59 

(33.85) 

(5.7i) 

(13.99) 

(26.12) 

(24.61) 

-2.94 

12.21 

21.31 

-5.78 

6.20 

(53.43) 

(86.16) 

(115.20) 

(87.90) 

(88.91) 

Pitch 
Manual 

Roll 
Manual 

Automatic 

Across 
Control 

Task _ _ _ | _  _ 

*  Mean  difference  in  feet  between  failure  insertion  and  response. 
**  One  standard  deviation. 

***  An  interaction  of  roll  manual  control  with  monitor  modes  under¬ 
lies  these  data:  See  also  Table  R-43. 

Note:  Means  contained  within  the  box  are  not  significantly 
different. 


The  addition  of  the  caution  light  drastically  reduced  average  lateral 
displacement  change  during  radar  altimeter  failures  when  roll  was  manual 
(from  no  monitor  154  feet  to  1,97  feet),  but  created  a  large  average  change 
in  lateral  displacement  during  HSI  failures  (from  no  monitor  2.75  feet  to  87 
feet).  The  magnitude  of  lateral  displacement  changes  during  these  worst 
case  conditions  deserves  some  diagnosis. 

Table  R-44  shows  the  roll  manual,  radar  altimeter,  no  monitor  and  cau¬ 
tion  light,  HSI  failure  data  two  ways.  First,  the  average  change  data 
treatment  repeats  what  was  previously  shown  in  Table  R-43.  The  change  in 
absolute  lateral  displacement  (signs  ignored)  for  the  same  conditions  rep¬ 
resent  the  second  treatment. 

In  review,  the  average  change  subtracts  the  lateral  displacement  at 
pilot  response  from  the  lateral  displacement  at  failure  insertion.  If  Y  = 
lateral  displacement  in  feet,  the  +  Y  would  occur  right  of  the  centerline, 

-Y  would  occur  left  of  the  centerline,  and  AY  =  Y  failure-  -Y  response.  Any 
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positive  value  of  Ay  would  therefore  represent  aircraft  movement  from  right 
to  left  whether  or  not  it  crossed  the  centerline.  Negative  values  of  Ay 
represent  movement  from  left  to  right. 


TABLE  R-44 

Lateral  Displacement  Change  During  Two  Display  Failure  Conditions 


Data 

Treatments 

Roll  Manual 

Average 

Absolute 

Worst  Conditions 

Change 

Change 

No  Monitor  with 

RA  failure 

+  154  * 

-  3 

Caution  with 

H3I  failure 

+  57 

-  43 

*  Feet, 


In  both  roll  manual  cases  shown  in  Table  R-44, the  change  indicated  air¬ 
craft  movement  to  the  left;  but,  what  we  really  want  to  know  is,  were  the 
pilots  making  their  lateral  situation  better  or  worse?  They  could  have  been 
laterally  displaced  at  the  failure  insertions  and  have  been  converging  on 
the  centerline  of  the  localizer;  or,  they  could  have  been  on  the  beam  at 
failure  insertion  and  have  been  diverging  at  response.  Since  the  absolute 
average  changes  ( | A  Y I  =  I  Yf  I  -  I  Y  i  )  were  negative,  the  absolute 
values  of  lateral  displacement  at  responSe  were  larger  than  the  absolute 
values  at  failure  insertion.  Therefore,  the  amount  of  lateral  displacement 
during  roll  manual  radar  altimeter  failures  with  no  monitor,  and  IIS I  failures 
with  caution  light  actually  changed  from  an  error  on  the  right  side  of  the 
course  to  a  larger  error  on  the  left  side  of  the  course  as  shown  in  Figure  R-l. 
If  lateral  drift  was  building  at  the  response,  the  lateral  displacement  could 
result  in  a  marginal  situation  below  200  feet  of  altitude. 
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154/2  =  77 
3/2  =  1.5 
Failure  =+75.5' 
15)+lReBponBe=-78.5’ 


Roll  Manual,  RA  Failure,  No  Monitor 


87/2  =  43. 5 
43/2  =  21.5 

Failure  =+22.0' 
Response  =-65' 


Roll  Manual,  HSI  Failure,  Caution  Light 


FIGURE  R-l  -  Lateral  Displacement  Hiring  Two  Roll  Manual  Conditions 


Lateral  Drift.  The  monitor  mode  by  pilot  control  task  by  display  failure 
interactions  shewn  in  lateral  displacement  data  was  not  significant  in  the 
lateral  drift  data.  We  must,  therefore,  examine  the  available  evidence  as  a 
function  of  two  data  treatments  by  monitor  mode  and  control  task  in  order  to 
determine  the  seriousness  of  the  lateral  displacement  data  just  presented. 
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First,  Table  R-45  shows  that  the  average  lateral  drift  at  pilot  response  was 
lower  durin,  the  no-monitor  approaches  (-1,52  feet  per  sec.)  than  during  the 
caution-plus-annunciator  mode  (2.22  feet  per  sec.).  The  change  in  lateral 
drift  followed  the  same  pattern,  but  indicated  that  the  monitor-with -caution 
and  no -monitor  approaches  were  not  significantly  different;  very  small  values 
resulted.  The  lateral  drift  at  failure  insertion  was,  therefore*  the  same  as 
the  lateral  drift  at  pilot  response  for  the  no-monitor  and  vith-cauticn 
conditions.  Note,  however,  that  the  variability  of  these  data  was  rather  high — 
one  standard  deviation  was  better  than  10  feet  per  second  in  all  cases. 


TABI£  R-U5 

Lateral  Drift  During  Display  Failures 


Monitor  Mode 


Measure 

No 

Monitor 

With 

Caution 

Caution 
+  Annunciator 

At  Pilot 

-1.52* 

0.11 

2.22 

Response 

(12.1*5)** 

(13.60) 

(11.98) 

1 

Change  During 

0.76 

-0.64 

Failure 

-(11.01) 

(10.38) 

-2.66 

(10.55) 


*  Feet  per  second. 

**  One  standard  deviation. 

Note:  Means  contained  within  boxes  are  not  significantly  different. 
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Returning  now  to  the  roll  manual  lateral  problem.  Table  R-46  shows  an 
absolute  average  lateral  drift  at  pilot  response  of  21  feet  per  second.  The 
change  in  absolute  lateral  drift  during  the  failures  was  positive  (2.91  feet 
per  second),  Indicating  that  drift  was  smaller  in  absolute  magnitude  at 
pilot  response  than  at  failure  insertion.  Again,  the  standard  deviations 
were  large  enough  to  take  notice  of  them. 


TABIZ  R-46 

Lateral  Drift  During  Display  Failure 


Roll 

Roll 

Measure  Manual. 


At  Pilot  21.36* 

Response  (15.43)** 

Change  During  2.91 

Failure  (11. 6l) 


Pilot  Control  Task 


Pitch 

Manual 

Automatic 

5.35 

2.36 

(4.75) 

(2.11) 

-1.11 

-0.33 

(4.35) 

(1.84) 

*  Absolute  average  in  feet  per  second. 

**  One  standard  deviation. 

Note:  Means  contained  within  boxes  are  not  significantly  different. 


Summary.  Thus  far  it  appears  that  the  roll  manual  lateral  displacement 
during  HSI  and  radar  altimeter  failures  was  not  getting  any  worse,  i.e., 
Lateral  drift  was  high,  but  was  getting  smaller  at  pilot  response.  Pitch 
manual  and  automatic  approaches  show  much  better  lateral  performance,  as 
would  be  expected.  Finally,  the  caution  light  monitor  mode  reduced  lateral 
displacement  during  radar  altimeter  flag  failures,  quite  likely  by  bringing 
the  radar  altimeter  failure  to  the  attention  of  the  pilot.  During  roll 
manual  display  failures,  an  HSI  flag  was  a  go-around  situation,  and  the 
presence  of  the  caution  light  seemed  to  create  a  lateral  displacement  problem. 
Since  this  was  a  go-around,  it  is  possible  that  the  pilots  concentrated  on 
executing  the  go-around  in  pitch,  giving  lateral  displacement  performance  a 
secondary  priority.  This  hypothesis  was  somewhat  verified  by  heading  error 
data  showing  a  similar  effect. 
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Heading  error.  As  in  lateral  displacement  and  lateral  drift,  the  absolute 
heading  error  change  (Table  R-47)  shewed  an  average  increase  in  heading  error 
of  -1.75  degrees  during  roll  manual  HSI  failures.  An  increase  in  heading 
error  was  also  seen  during  vertical  gyro  failures  while  roll-manual  fl.19 
degrees).  Both  of  these  conditions  were  go-around  situations  out  of  roll 
manual  control,  and  seem  to  suggest  that  the  roll  axis  was  at  least  momentarily 
disregarded  while  executing  go-arounds.  Also  evident  was  a  reduction  in 
heading  error  while  roll  manual  with  ADI  flags  (not  a  go-around).  The  vari¬ 
ability  of  heading  error  was  in  the  4-degree  region  during  roll  manual  failures. 


TABLE  R-47 

Heading  Error  Change  During  Display  Failures 


Pilot 

Control 

Task 

Display  Failure 

Across 

Display 

Failures 

ADI 

HSI 

RA 

VGU 

Pitch 

0.03* 

-o.o4 

0.19 

0.07 

0.06 

Manual 

(1.42)#* 

(1.08) 

(1.38) 

(1.43) 

(1.36) 

Roll 

0.59*** 

-1.75*** 

0.26 

-1.19*** 

0.10 

Manual 

(4.71) 

(4.94) 

(4.28) 

(3.89) 

(4.54) 

Automatic 

-0.24 

-0.05 

-0.05 

-0.34 

-0.71 

(1.21) 

(1.11) 

(0.98) 

(1.25) 

(1.14) 

Across 

Control 

0.13 

-0.67 

0.13 

-0.49 

-0.21 

Task 

(2.93) 

(3.08) 

(2.64) 

(2.55) 

(2.82) 

*  Mean  difference  in  degrees  between  failure  insertion  and  response. 

#*  One  standard  deviation. 

*#*  These  scores  are  significantly  afferent  from  each  other  and  all  the  rest. 
Remaining  scores  are  not  significantly  different. 
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Roll  attitude.  The  only  roll  attitude  data  treatment  that  revealed  a  per¬ 
formance  difference  as  a  function  of  the  experimental  variables  was  the 
absolute  roll  attitude  at  pilot  response  shewn  in  Table  R-48.  When  roll  was 
manually  controlled,  there  was  an  average  4.6-degree  roil  attitude  at  pilot 
response.  The  change  in  roll  attitude  from  failure  insertion  to  pilot 
response  was  -O.36  degrees  on  the  average,  with  a  4.65-degree  standard  devi¬ 
ation.  The  change  in  absolute  roll  attitude  was  -0.21  degrees  on  the 
average  with  a  2.99"degree  standard  deviation.  Roll  attitule  at  pilot  res¬ 
ponse  was  high  in  magnitude  and  in  the  opposite  direction  from  roll  attitude 
at  failure  insertion. 


TABI£  R-48 

Absolute  Roll  Attitude  at  Pilot  Response 
to  Display  Failures 


Roll 

Pitch 

Manual 

Manual 

Automatic 

4.60* 

0.86 

0.42  1 

(4.95 )** 

(1.39) 

(0.91) 

*  Absolute  average  in  degrees. 

**  One  standard  deviation. 

Note:  Means  contained  vifchin  the  same  box  are  not  significantly  different. 


Sunmary  of  -performance  data.  The  full  monitor  system  yielded  the  best 
performance  for  all  pilot  control  tasks  (split-axes  and  automatic)  and  for 
all  display  failures  that  were  given.  Vertical  velocity  change  showed  the 
flag  light  to  be  of  some  benefit  when  the  annunciator  was  not  in  operation, 
but  performance  with  the  annunciator  panel  and  caution  light  was  better.  The 
roll  manual  pilot  task  yielded  large  lateral  displacement  changes  during 
radar  altimeter  and  HSI  failures  with  no  monitor,  and  during  the  caution 
light  monitor  mode  approaches.  Although  the  full  monitor  system  signifi¬ 
cantly  reduced  these  large  lateral  displacement  changes  (See  Table  R-49  ), 
the  best  performance  while  roll  manual  showed  an  average  11-foot  change  with 
a  102-foot  standard  deviation  during  display  failures.  The  possibility 
of  a  102-foot  lateral  displacement  change  at  low  altitude  was  therefore 
quite  high. 

Pitch  attitude  change  and  vertical  velocity  change  revealed  that  the 
pilots  were  initiating  go-around  prior  to  engaging  go-around  mode  or  dis¬ 
connecting  the  autopilot.  Lateral  displacement  change  during  display 
failures  suggested  that  the  pilots  disregarded  the  lateral  axis  momentarily 
during  the  initial  part  of  the  go-around.  Lateral  displacement  changed 
from  error  on  the  right  side  of  the  course  to  larger  error  on  the  left, 
quite  possibly  because  of  left-handed  pilot  wheel/column  operation  and  right- 
handed  thrust  application  during  go-around. 
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Concluding,  the  full  monitor  system  v&s  necessary  to  stabilize  performance, 
but  it  did  not  yield  extremely  precise  (especially,  lateral)  performance  vhen 
the  pilot  was  distracted  by  a  manual  control  task.  The  touchdown 
performance  measures  (in  a  following  section)  show  that  the  pilot  did  not 
always  recover  the  lateral  axis  prior  to  touchdown.  Go -around  performance, 
also  presented  in  a  following  section,  shows  lateral  displacement  also. 
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significant  at  the  indicated  level  of  confidence 


Flare  Mode  Failure 


During  three  approaches  for  each  subject  (54  approaches  total)  the  auto¬ 
pilot  flare  mode  failed  to  engage  at  50  feet.  When  this  happened,  the  only 
cue  the  pilot  had  was  that  the  mode  engagement  light  on  the  mode  progress 
display  failed  to  turn  on.  The  pilots  did  not  detect  the  flare  mode  engage¬ 
ment  failure  37  times  (69$).  Resulting  touchdown  performance  is  shown  in 
Table  R-51. 


TABLE  R-51 

Flare  Mode  Failure  Touchdown  Performance 


Measure 

Mean 

Standard 

Deviation 

Distance  Down  Runway 

1025.0  ft. 

297.0  ft. 

Vertical  Velocity 

464.0  ft. /min. 

160.0  ft. /min. 

Pitch  Attitude 

1.9  deg. 

1.1  deg. 

Roll  Attitude  * 

0.5  deg. 

1»0  d6^« 

Heading  Error  * 

1.5  deg. 

1.6  deg. 

Lateral  Displacement  * 

16.4  ft. 

19.7  ft. 

Lateral  Drift  * 

2.2  ft. /sec. 

4.6  ft. /sec. 

Airspeed 

127.8  kts. 

2.8  lets. 

*  Absolute  average  (signs  ignored). 


Note  in  Table  R-51  that  the  only  serious  consequence  of  the  flare  mode 
failure  was  that  the  vertical  velocity  on  touchdown  was  high,  as  would  be 
expected.  The  pilots  generally  thought  that  failure  of  a  light  to  turn  on 
was  a  poor  way  to  alert  the  pilot  to  a  problem,  especially  in  this  portion 
of  the  profile  approach,  where  vision  is  narrowing  down  to  essential  per¬ 
formance  parameters  on  the  pilot's  display  panel. 

Go -Around  Performance 

Go-around  performance  measurement  was  taken  at  the  lowest  altitude 
attained  during  the  go-around  execution.  Data  were  also  generated  that 
described  the  change  in  performance  between  the  time  the  pilot  responded  to 
the  failure  (pilot  response)  and  the  time  that  the  lowest  altitude  was 
attained.  The  analysis  of  variance  summary  is  shown  in  Table  R-56. 
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Vertical  plane  performance.  The  failure  monitor  mode  did  not  influence 
vertical  plane  performance  during  go- around;  but  pilot  task  prior  to  the  sec¬ 
ond  failure  that  created  the  go- around  requirement  did  influence  performance. 
The  lowest  altitude  was  attained  (l)  more  quickly,  (2)  in  less  longitudinal 
distance,  and  (3)  with  less  altitude  consumed  when  the  pilot  was  pitch  manu¬ 
al  split-axis  compared  to  roll  manual  split-axis  (Table  R-52).  Of  the  203 
go-arounds  that  occurred,  there  were  only  three  cases  during  which  the  simu¬ 
lator  descended  below  the  glide  slope  at  the  lowest  altitude.  The  worst 
case  was  a  1.23  foot  loss.  In  no  case  was  this  altitude  lost  during  go- 
arounds  with  the  full  monitor  system  In  operation.  In  fact.  Table  R-53  shows 
that  the  aircraft  was  19.0  feet  above  the  glides  lope  at  the  lowest  altitude 
on  the  average  during  go-arounds  with  the  full  monitor  system  in  operation. 
There  were,  however,  some  cases  of  smaller  altitude  gains  as  reflected  by  the 
11.3  foot  standard  deviation  of  the  scores  underlying  that  19.0  foot  perform¬ 
ance  mean. 

Curiously,  the  altitude  consumed  between  response  and  the  lowest  altitude 
attained  seemed  to  be  only  a  function  of  the  amount  of  altitude  remaining.  A 
scatter  plot  of  the  altitude  consumed  between  pilot  response  and  the  lowest 
altitude  attained  and  the  altitude  at  which  response  occurred  is  shown  as 
Figure  R-2.  Note  that  there  is  no  real  clustering  of  the  data  except  on  the 
zero  altitude  consumed  line  which  is  a  data  artifact.1  Non-clustering  of 
the  data  suggests  that  the  pilots  employed  a  strategy  of  using  the  available 
altitude  to  execute  their  go- around  rather  than  always  going  around  as  quick¬ 
ly  and  smoothly  as  possible.  This  strategy  is  alarming,  for  it  will  be  abso¬ 
lutely  safe  only  if  the  pilot  correctly  interprets  the  amount  of  altitude  he 
has  available  to  perform  the  maneuver.  Note  that  two  go-around  attempts 
struck  the  ground.  These  were  cases  where  the  pilot  decided  to  go-around  at 
20  feet  of  altitude  and  didn't  make  it.  Six  go-arounds  came  within  10  feet 
of  the  ground,  and  a  total  of  19  go-arounds  came  within  20  feet  of  the  ground. 
Since  an  altitude  loss  below  the  glide 8 lope  occurred  only  three  times,  most . 
approaches  would  have  been  within  obstacle  clearance  limits,  PROVIDED  that 
their  lateral  deviation  was  not  excessive  when  they  were  close  to  the  ground. 
Lateral  performance,  however,  shows  lateral  error  accrual  during  go-around 
under  certain  conditions  that  could  be  serious  when  coupled  with  the  vertical 
plane  go-around  strategy  employed. 


1.  The  37  cases  of  zero  altitude  consumed  was  a  data  artifact  because  the 
measurement  system  required  that  the  pilot  (l)  disconnect  the  autopilot, 
(2)  trigger  go-around  mode  or  (3)  engage  the  "response"  bar  to  indicat  - 
response  to  a  failure.  In  37  cases,  the  pilots  had  already  attained 
positive  vertical  velocity  before  they  exerted  any  of  the  actions  above 
to  call "response". 
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Insertion  Versus  Altitude 
nd  Lowest  Altitude 


TABLE  R-52 


Go-  Around  Performance  aa  a  Function  of  Pilot  Task  Prior  to  Second  Failure 


Measure  at 

Lowest  Altitude 

Pilot  Task 

Pitch  Manual  Roll  Manual 

Across 

Pilot  Task 

Tine  from  Pilot 

3.9  sec.  * 

4.5  sec. 

4.1  sec. 

Response 

(2.0  sec.)  ** 

(2.0  sec.) 

(2.0  sec.) 

Distance  from  G/S 
Transmitter 

n.s.  *** 

n.s. 

1608.0  ft. 
(1341.0  ft.) 

Distance  from 

785.0  ft. 

1063.0  ft. 

905.0  ft. 

Pilot  Response 

(66i.o  ft.) 

(905.0  ft.) 

(734.0  ft.) 

Average  Lowest  Altitude 

n.s. 

n.s. 

88.8  ft. 
(74.1  ft.) 

Altitude  Consumed  from 

31.7  ft. 

43.9  ft. 

37.0  ft. 

Pilot  Response 

(33.1  ft.) 

(37.7  ft.) 

(35.6  ft.) 

Altitude  Consumed  from 
Failure  Insertion 

n.s. 

n.s. 

125.8  ft. 
(74.7  ft.) 

Pitch  Attitude 

n.s. 

n.s. 

7.7  deg. 
(3.9  deg.) 

Indicated  Airspeed 

133.4  kts. 

(6.0  kts.) 

136.3  kts. 
(11.3  kts.) 

134.7  kts. 
(8.9  kts.) 

Roll  Attitude 

1.7  deg. 

(2.1  deg.) 

6.9  deg. 

(5.8  deg.) 

4.0  deg. 
(5.8  deg.) 

Lateral  Drift 

3.5  fp«. 

(6.1  fps.) 

23.2  fps. 
(17.6  fps.) 

12.0  fps. 
(15.8  fps.) 

*  Average  performance. 

**  One  standard  deviation, 

***  No  significant  difference  between  pitch  and  roll  manual  scores. 


90 


TABUS  R-53 


Altitude  Gain 


Monitor  Mode 

Across 

No 

Caution  + 

Monitor 

Monitor 

Caution 

Annunciator 

Mode 

13.7  * 

16.1 

19.0 

16.4 

(9.3)  ** 

(10.6) 

(11.3) 

(16.5) 

*  Average  in  feet. 

**  One  standard  deviation. 


Lateral  plane  performance.  Lateral  displacement  during  pitch  manual 
split-axis  approaches  was  no  particular  problem,  but  lateral  displacement 
(Table  R-54),  heading  error  (Table  R-55)  and  lateral  drift  (Table  R-52) 
show  that  the  aircraft  was  off  of  the  localizer  course,  and  departing  at  a 
high  (23  feet  per  second)  rate  at  the  lowest  altitude  when  roll  manual  split- 
axis  conditions  existed  prior  to  the  second  failure.  These  data  corroborate 
the  lateral  performance  data  seen  earlier  during  display  failures;  the  pilots 
tend  to  disregard  roll  in  favor  of  pitch  during  go-arounds.  In  and  of  itself, 
this  strategy  is  not  alarming.  When  one  couples  the  roll  performance  and 
strategy  with  the  pitch  axis  strategy  of  using  the  available  altitude  to  exe¬ 
cute  the  go-around,  a  potential  hazard  is  indicated. 


TABLE  R-54 

Lateral  Displacement  at  Lowest  Altitude  During  Go-around 


Monitor  Mode  Across 

Pilot  No  Caution  +  Monitor 

Task  Monitor  Caution  Annunciator  Mcxie 


Pitch 

Manual 


18.28  *  14.87  23.54  18.89 

(30.20)  **  (19.37)  (48.34)  (34.76) 


Roll  [257.52  206.67I  126.21 

Manual  (176.12)  (194,42)  (125.89) 


187.96 

(171.96) 


Across  114.82  95.91  71.45  92.18 

Pilot  (163.54)  (158.02)  (105.70)  (143.08) 

Task. _ _ 

*  Mean  lateral  displacement  in  feet. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly 
different. 
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If  the  pilot  simult  aneously  allows  the  aircraft  to  go  lew  and  at  the 
same  time  drift  off  course  laterally,  he  could  depart  the  obstacle  clearance 
zone.  One  may  scoff  at  the  alarm,  but  remember  that  the  pilot  could  be  just 
a  few  feet  above  the  glideslope,  a  few  feet  above  the  ground  during  go-around, 
and  conceivably  strike  the  glideslope  antenna  in  the  two-three  standard 
deviation  case  of  roll  manual  lateral  displacement  shewn. 


TABUS  R-55 

Heading  Error  at  Lowest  Altitude  During  Go- Around 


Monitor  Mode 

AoroMB 

Pilot 

Ta3k 

No 

Monitor 

Caution 

Caution  +  Monitor 

Annunciator  Mode 

Pitch 

1.03  * 

►  [3.66 

3.60 

2.86 

Manual 

(0.85) 

**  L.&93 

(1.83)_ 

(2.02) 

Roll 

1  8.68 

8.l4 

6.86 

7.77 

Manual 

1(7.52) 

(6.99) 

(5.88) 

(6.69) 

Across 

4.11 

5.56 

5.12 

4.99 

Pilot 

Task 

(6.08) 

(5.23) 

(4.51) 

(5.25) 

*  Mean  heading  error  in  degrees. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly 
different. 


Summary  of  go- around  performance.  The  vertical  and  lateral  plane  go- 
around  strategy  of  the  pilots  seen  in  Jiis  study  needs  to  be  verified  when 
pilots  go-around  out  of  automatic  approaches  (Phase  III  upcoming)  and  in 
real  aircraft  under  adequately  controlled  experimental  conditions.  At  the 
present  time  one  must  conclude  on  the  basis  of  these  data  that  pitch  manual 
split-axis  control  leads  to  safer  go- around  performance  than  roll  manual 
split-axis  control,  and  that  the  full  monitor  system  yielded  better  lateral 
performance  than  no  monitor  during  the  roll  manual  split-axis  conditions 
prior  to  go- around  (the  worst  case). 
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TABIE  R-56 


Go-Around  Performance  Analysis  of  Variance  Summary 


Measure  at 

Lowest  Altitude 

Source  of  variance 

Pilot  Task  (P) 

Failure  Mode  (F) 

RxF 

Time  from 

Pilot  Response 

.05* 

** 

Distance  from  G/S 
Transmitter 

Altitude  Loss 
(or  Gain) 

.05 

Lowest  Altitude 

Altitude  Consumed  from 
Pilot  Response 

.05 

Altitude  Consumed  from 
Failure  Insertion 

Distance  Consumed  from 
Pilot  Response 

.05 

Pitch  Attitude 

Indicated  Airspeed 

.05 

Roll  Attitude 

.01 

Lateral  Drift 

.01 

Heading  Error 

.01 

Lateral  Displacement 

.01 

.01 

*  Source  of  variance  significant  beyond  the  level  of  confidence 


indicated. 

**  Blank  spaces  indicate  non-significant  source  of  variance. 


p 
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Touchdown  Performance 


Winde hears  were  confounded  with  failure  modes  in  the  experimental  design 
because  it  was  assumed  that  *n  failures  would  have  been  detected  and  acted 
upon  prior  to  the  aircraft  entering  the  shearwinds  below  100  feet  of  altitude. 
Unfortunately,  many  failures  were  not  detected  in  the  no  monitor  and  caution 
light  monitor  modes.  No-monitor  was  confounded  with  the  pure  tailwind  shear; 
caution  was  confounded  with  the  lateral  shear;  and,  caution-plus- annunciator 
was  confounded  with  the  combination  tailwind  and  lateral  shears.  The  effects 
of  windshears  upon  touchdown  may  be  influenced,  in  part,  by  the  monitor  mode 
performance.  This  is  the  first  problem  of  touchdown  performance  interpretation 

A  second  problem  of  touchdown  data  interpretation  arose  because  there  were 
unequal  numbers  of  landings  under  all  experimental  conditions.1  Roll  manual 
data  is  based  upon  only  32  actual  landings.  Pitch  manual  data  is  based  upon  52 
actual  landings.  The  autopilot,  on  the  other  hand,  was  responsible  for  162 
landings.  From  a  statistical  standpoint,  performance  estimates  based  upon 
162  scores  are  far  more  reliable  than  those  based  upon  32  scores. 

The  following  analysis  and  interpretation  of  touchdown  performance,  there¬ 
fore,  must  be  tempered  (qualified)  by  a  knowledge  of  these  two  problems. 

Touchdown  performance  shown  in  Table  R-57  did  not  look  promising.  Overall, 
50.4 $  of  the  246  scoreable  touchdowns  were  within  the  allowable  tolerances 
indicated  in  Table  R-58.  Out  of  32  roll  manual  landings,  6.2$  were  successful; 
17.3$  of  the  pitch  manual  landings  were  successful;  69.8$  of  the  automatic 
landings  were  successful.  A  breakdown  of  percent  successful  landings  by  indi¬ 
vidual  parameter  is  included  in  Table  R-59*  Also  shown  is  the  percent  success¬ 
ful  touchdowns  (l)  when  six  out  of  the  seven  criteria  were  met  and  (2)  when 
five  out  of  the  seven  criteria  were  met  on  each  touchdown. 


1.  A  larger  number  of  touchdowns  was  planned  in  the  experimental  design.  At 
least  one  failure  always  occurred  prior  to  touchdown.  Many  subjects, 
however,  conservatively  executed  missed-approaches  when  they  could  have 
continued  under  the  decision  rules  set  forth  for  this  experiment.  It  was 
near  impossible  to  achieve  the  munber  of  touchdowns  that  were  planned. 
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TABLE  R-57 


Percent  Successful  Touchdowns 


Windshears 

Pilot  Task 

Across 
Pilot  Task 

Pitch 

Manual 

Roll 

Manual 

Automatic 

Tailwind 

17.6* 

9.1* 

85.2* 

6l.O* 

(17)  * 

(11) 

(54) 

(82) 

Tailwind  & 

11.1* 

9.1* 

66.7* 

47.0* 

Lateral 

(18) 

(11) 

(54) 

(83) 

Lateral 

23.5* 

0.0 * 

57.4* 

43.0* 

(17) 

(10) 

(54) 

(81) 

Across 

17.3* 

6.2 * 

69.8* 

50.4* 

Windshears 

(52) 

(32) 

(162) 

(246) 

*  Number  of  actual  . ouchdovms  under  these  conditions  that 
could  be  scored. 


TABLE  R-58 

Criteria  for  Successful  Touchdowns 


Measure 

Tolerance  Band 

Percent  of  Time 
Within  Tolerance 

Distance  Down  Runway 

0-3000  ft. 

94.3 

Lateral  Displacement 

+  60  ft. 

82.5 

Pitch  Attitude 

0-5  deg. 

96.3 

Roll  Attitude 

+  5  deg. 

94.7 

Heading  Error 

+  4  deg. 

80.9 

Lateral  Drift 

+  9  fps. 

t'3.7 

Vertical  Velocity 

O-360  fpm. 

78.5 
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TABI£  R-59 

Successful  Touchdown  by  Channel 


2  criteria  not  laet 
1  criteria  not  met 
All  criteria  met 


2  criteria  not  met 
1  criterion  not  met 
All  criteria  met 


Automatic  X  Range 

Y  Displ 
Pitch 
Roll 
Heading 

Y  Drift 


100.0 

96.3 

96.3 

98.1 

100.0 

96.3 


96.3 

81.5 
98.1 

94.4 
83.3 

92.6 
_ 87*0 


2  criteria  not  met 

100.0 

94.4 

1  criterion  not  met 

98.1 

87.0 

All  criteria  met 

85.2 

57.4 

Winds 

X  Range 

93.9 

92.6 

Over 

Y  Displ 

87.8 

76.5 

Pilot 

Pitch 

93.9 

98.8 

Task 

Roll 

96.3 

90.1 

Heading 

90.2 

75.3 

Y  Drift 

85.4 

79.0 

Altitude 

80,5 _ 

80.2 

2  criteria  not  met 

89.0 

85.2 

1  criterion  not  met 

81.7 

74.1 

All  criteria  met 

61.O 

43.2 

98.1 
92.6 

100.0 

100.0 

85.2 
94.4 
.8.7*0. 
98.1 

92.6 

66.7 


98.1 

90.1 

98.1 

97.5 

89.5 

94.4 
.20*1, 

97.5 

92.6 
69.8 


.2  91.6 

.1  77.1 

.2  47.0 
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An  analysis  of  individual  performance  measures  (Table  R-60)  revealed 
that  pilot  task  (manual  roll,  manual  pitch,  or  automatic)  was  a  significant 
variable  for  all  measures.  Significant  wind  conditions  occurred  next  most 
often  and  there  were  two  measures  in  which  a  significant  wind  by  pilot  task 
interaction  occurred.  An  examination  of  these  data  follows: 


TABLE  R-60 

Touchdown  Performance  Analysis  of  Variance  Summary 


Touchdown 

Source 

of  Variance 

Performance 

Measure 

Pilot 

Task(p) 

Wind  (W) 

PxW 

Distance  from 

G/S  Transmitter 

.01* 

.01 

Indicated  Airspeed 

.01 

Pitch  Attitude 

.01 

.05 

Vertical  Velocity 

.01 

.05 

Roll  Attitude 

.01 

.01 

.01 

Heading  Error 

.01 

.01 

Lateral  Drift 

.01 

Lateral  Displacement 

.01 

*  Source  of  variance  significant  beyond  the  level  of 


confidence  indicated. 

Note:  Blank  spaces  indicate  non-significant  sources  of 
variance . 


Tailwind  shear.  This  condition  started  with  a  10-knot  headwind  shearing 
to  calm  from  100  feet  to  the  ground.  The  tailwind  shear  created  pitch  manual 
control  problems  during  the  flare.  An  average  1.1-degree  pitch  attitude  at 
touchdown  (Table  R-6l),  25  feet  ahead  of  the  glideslope  shack  with  a  vertical 
velocity  (Table  R-c2)  in  excess  of  450  feet  per  minute  suggest  that  the  pilot 
was  unable  to  flare  the  aircraft.  Under  these  conditions  the  flare  computa¬ 
tion  was  difficult  to  follow  because  the  pitch  steering  bar  did  not  command 
a  noticeable  change  until  below  20  feet.  By  the  time  the  pilot  recognized 
the  command  it  was  too  late  to  take  effective  control  action. 
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TABLE  R-6l 

Vertical  and  Longitudinal  Touchdown  Performance 


Pilot  Task  Winds 

Touchdown  — . .  • — . 

Performance  Pitch  Roll  Tailwind  W/Lat.  Lateral  Overall 

Measure  Manual  Manual  Auto  Shear  Shear  Shear  Average 


Distance 
Down  Runway 

975* 

1132 

1397 

in  Ft. 

[(1395)**  (1324)1 

j  (543) 

Indicated 

Airspeed 

127.7  128.1 

125.6 

in  Knots 

(4.0)  (11.0) 

(2.8) 

Pitch 

Attitude 

l.i 

3.5 

3.8 

in  Degrees 

(1.8)  j 

(2.0) 

(1.2) 

ll4o 

1155 

1530 

1273 

(822) 

(865) 

(1030) 

(923) 

n.s. 

126.4 

(5.0) 

3.1 

37P 

2.6 

2.9 

(1.7) 

(1.8) 

(1.5) 

(1.7) 

*  Means. 

**  One  standard  deviation. 

*** Winds  did  not  significantly  influence  indicated  airspeed. 

Note:  Means  contained  within  the  Same  box  are  not  significantly  different. 


TABLE  R-62 


Vertical  Velocity  at  Touchdown 


Windshears 

Pitch 

Manual 

Pilot  Task 

RoU 

Manual 

Aorosr. 

Pilot 

Automatic  Task 

Tailwind 

-498* 

-327 

-206 

-283 

1 

(1M)** 

(226; 

(106) 

(258) 

Tailwind  & 

-474 

-182 

-223 

-272 

Lateral 

(290) 

(142) 

(127) 

(205) 

Lateral 

-271 

-261 

-228 

-241 

(330) 

(194) 

(170) 

(213) 

Across 

-415 

-256 

-219 

-265 

Windshears 

(317) 

(194) 

(136) 

(226) 

*  Mean  vertical  velocity  in  feet  per  minute. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  same  box  are  not  significantly 
different. 
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Lateral  shear.  The  lateral  shear  was  cr»  cd  by  shearing  a  10  knot  90 
degree  crosswind  to  calm  as  the  aircraft  des-.  ended  from  100  feet  to  touch* 
down.  Lateral  shear  mainly  affected  the  lateral  measures,  especially  during 
roll  manual,  split -axis  control. 

Roll  attitude  at  touchdown  was  the  highest  (6.3  degrees)  during  roll 
manual  landings  with  the  lateral  wind shear  (Table  R-64).  Heading  error  in 
Table  R-63  was  greater  during  lateral  shears  than  during  tailwind  shears  as 
would  be  expected.  Curiously,  the  shearwinds  did  not  significantly  influence 
lateral  drift  or  lateral  displacement.  Lateral  drift  and  displacement  were 
systematically  influenced  by  pilot  control  tasks.  The  excessive  lateral 
deviation  (175  feet)  and  lateral  drift  (21.8  feet  per  second)  during  roll 
manual  touchdowns  apparently  obscured  any  small  deviations  created  by  the 
windshear.  These  data  serve  as  further  evidence  that  the  lateral  control 
problem  during  manual  instrument  landings  is  severe. 


TABLE  R-63 

Lateral  Touchdown  Performance 


_ Pilot  Task  _ Winds _ 

Performance  Roll  Pitch  Tailwind  W/Lat.  Lateral  Overall 

Measure _ Manual  Manual  Auto  Shear  Shear  Shear  Average 


Heading 

Error  in 

7.5  * 
(6.7)** 

2.2 

(1.9) 

1.7 

(2.3) 

1.6 

(3.0) 

2.7 

(3.7) 

3.2 

(4.1) 

2.5 

(3.7) 

Degrees 

Lateral 

Drift  in 
ft.  per  sec. 

21.8 

(16.9) 

4.6 

(5.8) 

2.4 

(5.1) 

n.s  *** 

5.4 

(10.1) 

Lateral  175.0  132.0 

Displacement  (186.2)  .f63.3) 

25.6 

(53.4) 

n.s 

46.4 

(97.9) 

*  Mean 

**  One  standard  deviation. 

***  Winds  did  not  significantly  influence  lateral  drift  and  lateral 
displacement . 

Note:  Means  contained  within  the  same  box  are  not  significantly  different. 


* 
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TABI£  R-64 


Roll  Attitude  at  Touchdown 


Pilot  Task 

Across 

Roll 

Pitch 

Pilot 

Windshears  Manual 

Manual 

Automatic  Task 

Tailwind  1  2.7 

1 

i  0.3 

0.6- 

0.8 

*2.0) 

i 

:  (o.4) 

(1.1) 

(1.4) 

Tailwind  &  :  2.0 

1 

0.5 

0.7 

0.8 

Lateral  ;(2.6) 

1 

i 

I  (0.6) 

(0.9) 

(1.3) 

Lateral  6,3  * 

0.2 

1.2 

1.6 

(4.6)  **  | 

l(0.4) 

(2.4) j 

(3.1) 

Across  3.6 

0.3 

0.8 

1.1 

Windshear  (3.6) 

(0.5) 

(1.6) 

(2.1) 

*  Average  (absolute)  roll  attitude  in  degrees. 

**  One  standard  deviation. 

Note:  Means  contained  within  the  sane  box  are  not  sig¬ 
nificantly  different. 


Tailwind  shear  plus  lateral  shear.  Hie  combination  of  tailwind  and  lat¬ 
eral  shear  did  not  significantly  change  vertical  plane  performance  from  the 
pure  tailwind  shear  condition.  Neither  did  the  combination  significantly 
alter  lateral  plane  performance  from  the  pure  lateral  shear  condition. 

Final  comment  on  touchdown  performance.  This  study  was  a  severe  test. 
Pilots  were  asked  to  continue  in  manual  control  after  they  had  experienced 
one  control  failure.  Not  all  of  the  control  failures  were  detected  during 
approaches  that  had  less  than  the  full  monitor  system  in  operation.  Even 
with  one  control  failure  having  occurred,  display  failures  were  apt  to  occur. 
After  39  approaches  under  these  conditions  the  pilots  were  so  geared  to  look¬ 
ing  for  failures  that  their  manual  control  capability  suffered  when  (and  if) 
they  arrived  at  the  flare.  Under  these  stressed  conditions  the  tested  flight 
director  and  computers  did  not  make  the  man  effective. 

The  flight  director  system  and  human  pilot  could  not  handle  the  tailwind 
shear.  In  roll,  the  flight  director/pilot  combination  could  not  successfully 
handle  the  lateral  plane.  Since  human  pilots  have  been  visually  landing  air¬ 
craft  for  many  years,  it  can  only  be  concluded  that  either  the  proper  infor¬ 
mation  for  the  human  pilot  to  land  the  aircraft  is  not  yet  on  the  instrument 
panel,  or  the  information  that  is  available  is  not  being  used  effectively. 
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The  next  study  being  undertaken  by  the  STIR  project  will  more  carefully 
analyze  the  manual  flare  in  an  attempt  to  diagnose  the  manual  flare  and 
touchdown  problem.  Completely  removing  the  human  pilot  from  the  control 
loop  is  one  solution  to  the  problem.  An  alternative  solution  could  well 
come  from  an  understanding  of  why  the  pilot  cannot  adequately  control  the 
flare  (ver.ically  and  laterally)  with  existing  information. 

Rollout 

Following  each  touchdown,  the  pilot  had  to  deploy  the  spoilers,  push  the 
column  full  forward  (the  First  Officer  held  it  there  once  it  had  been  posi¬ 
tioned),  reverse  thrust,  apply  brakes,  and  steer  the  simulator  down  the  run¬ 
way  using  raw  heading  and  localizer  information  on  the  horizontal  situation 
indicator.  During  the  execution  of  the  study,  it  became  obvious  that  the 
task  was  unreasonable  in  the  simulator.  The  rollout  data  were  dropped  from 
the  analysis  for  the  following  reasons: 

The  task  was  difficult  in  and  of  itself.  The  pilot's  view  of  the  HSI  was 
partially  blocked  by  the  wheel  hub  when  the  column  was  held  full  forward;  the 
pilots  had  to  look  around  the  wheel  hub  while  reversing  thrust  and  braking. 

If  the  pilot  looked  over  the  top  of  the  wheel  hub,  the  upper  lip  of  the  HSI 
case  would  obscure  the  heading  index  and  the  top  portion  of  the  compass  card. 
Adding  to  the  difficulty  of  this  task,  the  rollout  aynamics  of  the  Simula  to-1" 
became  suspect  after  the  first  few  subject  pilots  were  run;  the  pilots  were 
consistently  going  divergently  unstable  and  leaving  the  runway.  It  was 
decided,  therefore,  to  drop  the  rollout  data  from  consideration  until  a  more 
satisfactory  simulation  of  rollout  could  be  achieved.  The  simulator  roll¬ 
out  dynamics  were  worked  on  for  the  remainder  of  the  study. 

General  Characteristics  of  Fault  Warning  Systems 

Although  only  one  fault  warning  system  was  instilled  in  the  simulator  for 
this  study,  various  modes  of  operation  were  employed  that  allow  some  inquiry 
into  general  features  of  fault  warning  systems.  In  addition  to  the  perfor¬ 
mance  data  already  presented,  there  was  a  wealth  of  pilot  opinion  that 
evolved.  This  section  will  examine  some  fault  warning  system  characteristics 
within  the  guidelines  of  this  study,  and  offer  pilot  comment. 

Caution  light.  When  asked  if  it  was  necessary  to  have  both  the  caution 
light  and  the  mechanical  flags  during  display  failures,  50$  of  the  pilots 
said  yes,  50$  said  no.  Thirty  percent  of  the  pilots  thought  that  the  flags 
wesetoo  small.  Thirty-five  percent  of  the  pilots  thought  that  the  radar 
altimeter  flag  needed  to  be  backed  up  by  the  caution  light.  Generally, flags 
outside  of  the  ADI  were  difficult  to  see. 

Flag  light.  Seven  out  of  nine  pilots  who  saw  the  flag  light  thought 
that  it  was  of  significant  value.  It  was  generally  agreed  that  such  a 
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light  should  indicate  flags  down  on  both  sides  of  the  cockpit.  Five  pilots 
thought  the  flag  light  should  be  above  the  ADI,  and  four  thought  it  ought 
to  be  in  a  central  location. 

Annunciator  panel.  Forty-five  percent  of  the  pilots  thought  that  it  was 
necessary  for  the  annunciator  panel  to  be  redundant  with  mechanical  flags 
with  the  present  system.  Given  a  flight  director  system  and  radar  altimeter 
display  with  larger  flags,  30^  of  the  pilots  could  classify  this  feature  as 
“necessary. " 

Discussion.  The  perform nee  data  showed  significantly  better  perfor¬ 
mance  in  the  caution  plus  annunciator  mode.  Flags  and  annunciator  without  the 
caution  light  were  not  investigated.  At  this  time  it  is  not  known  whether  the 
annunciator  caused  better  performance,  or  whether  better  performance  was  due  to 
the  combination  of  caution  and  annunciator.  The  small  flags  on  the  ADI  itself 
could  have  contributed  to  the  problem.  A  study  (Phase  III)  is  being  designed 
to  more  fully  look  at  the  necessity  of  redundancy  between  the  display  flags 
and  the  annunciator  panel. 

This  study  did  not  look  into  the  possibility  of  &  comparator  monitor  across 
the  two  sides  of  the  cockpit.  Given  only  a  comparator,  the  comparison  function 
may  need  annunciation.  For  example,  it  is  possible  for  two  vertical  gyros  to 
disagree  even  though  neither  one  shows  a  flag.  With  self -monitors,  the  self¬ 
monitor  could  assess  signal  quality  and  just  as  easily  trip  a  mechanical  flag 
as  illuminate  a  light.  The  saving  would  be  in  panel  space  and  some  increase 
in  warning  mechanism  reliability  (a  light  is  not  as  reliable  as  a  flag).  This 
saving  would  be  substantial  IF  FLAGS  WERE  SHOWN  TO  BE  AS  EFFECTIVE  AS  LIGHTS. 

Deviation  light.  The  general  consensus  of  the  pilots  was  that  the  deviation 
light  was  more  of  a  nuisance  than  a  help.  It  illuminated  when  excessive  devia¬ 
tion  was  detected.  This  was  just  the  time  when  the  pilots  were  usually  quite 
busy  trying  to  recover  the  course.  At  this  critical  time,  the  presence  of  a 
new  light  on  the  panel  forced  them  to  stop  paying  attention  to  what  they  were 
doing  in  order  to  read  th-  new  light.  It  compounded  the  problem.  Most  pilots 
knew  that  they  were  excessively  deviating  and  were  trying  their  best  to  do 
something  about  it. 

Ahn-rfr  enmman^.  An  abort  command  was  not  incorporated  into  the  fault  warn¬ 
ing  system  exercised,  but  decision  rules  were  given  to  the  pilots.  The  rule 
was  to  execute  go- around  if  a  display  failure  occurred  in  a  manually  controlled 
axis.  When  asked  if  they  developed  any  general  rules  of  thumb  for  executing 
decision  in  this  study,  six  of  the  22  pilots  (including  pre-experimental)  said 
"no."  Five  pilots  said  that  they  would  go-around  if  they  were  not  positive. 
Seven  pilots  cited  a  great  deal  of  confusion  generated  by  the  system  and  the 
failures,  but  pointed  out  that  farther  training  on  the  system  would  certainly 
help.  Four  pilot  responses  are  quoted  in  the  following  paragraph;  words  in 
parentheses  are  added  by  the  authors: 
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"After  flying  11-12  hours  non-stop  from  London-IAX,  I'm  too  tired  to 
correctly  apply  any  rule  of  thumb.  Even  subtle  failures  must  be  unmis- 
takeable  and  instantly  displayed  with  the  information  that  the  approach  can 
or  cannot  be  continued."  Another  responded,  "I  do  not  believe  that  our 
crews  today  making,  say  one  Cat  III  actual  per  year,  and  say  two  practice 
approaches  can  maintain  proficiency  in  making  last  minute  decisions  whether 
to  continue  or  go-around.  By  this  I  mean,  roll  axis  out,  (pitch)  autopilot 
O.K.  Now  DG  out,  should  I  continue?  Any  functions  inoperative  would  be 
enough  for  me  (to  go-around)."  Yet  another  said,  "I  mode  several  incorrect 
decisions  based  on  the  information  presented,  usually  due  to  too-hasty 
action.  I  found  it  difficult  to  'take  my  time'  when  a  failure  occurred  at 
100  feet."  A  fourth  pilot  said,  "Any  failure  below  200  feet  would  require  a 
go-around  as  far  as  I'm  concerned.  From  this  point  to  touchdown  (there)  is 
not  time  to  have  to  analyze  a  problem." 

The  responses  in  the  preceding  paragraph  seem  to  suggest  that  a  go-around 
or  abort  advisory  function  is  needed.  Yet,  when  asked  directly  if  they 
would  rather  have  a  decision  made  for  them  or  be  told  only  the  specific 
failures,  12  pilots  wanted  the  decisions  left  up  to  them.  The  principal 
reason  was  that  decisions  would  have  to  be  pre-programmed  and  there  are  too 
many  variables  to  cover  all  situations.  Some  pilots  suggested  that  the  seri¬ 
ousness  of  the  failure  be  indicated.  The  responses  of  four  pilots  are 
quoted  in  the  next  paragraph.  These  four  pilots  seemed  to  clarify  the  problem. 

One  pilot  said,  "Up  to  some  minimum  altitude  I  would  rather  have  the 
prero^fcLve.  If  a  failure  occurs  below  that  altitude,  I  would  rather  have  a 
simple  'abort'  light  and  a  fault  warning  system  to  help  analyze  the  situation 
at  a  safe  altitude  after  missed  approach."  A  second  pilot  said,  "Certain 
failures  at  100  feet  or  less  must  b«!  made  evident  to  the  pilot  in  such  a  way 
that  the  decision  to  abort  or  continue  is  made  for  him.  I  include  flare 
capability,  runway  alignment,  and  excessive  roll  or  pitch."  A  third  pilot 
said,  "If  the  command  is  reliable  it  would  be  invaluable.  Especially  from 
200  feet  to  touchdown.  Any  malfunction  in  this  portion  of  the  approach  does 
not  leave  enough  time  for  ground  school  prior  to  runway  contact."  A  fourth 
pilot  said,  "If  failures  progress  to  a  point  where  approach  is  hazardous, 
then  some  method  of  advising  decision  to  abort  (should  be  incorporated) 
instead  of  correlating  several  failures  to  determine  what  decisions  to  make." 

It  is  clear  that  the  question  should  have  been  limited  to  below  200  feet; 
it  is  the  time  factor  that  creates  the  interpretation  problem.  It  seems 
reasonable  that  pilot  acceptance  of  a  well -programmed  abort  advisory  below 
200  feet  would  be  good.  Such  an  advisory  would  not  be  without  problem, 
however,  because  one  can  think  of  an  exception  to  every  rule.  For  example, 
with  loss  of  glideslope  at  55  feet  of  altitude,  the  aircraft  would  be  into 
the  flare  mode  and  not  need  the  glideslope  before  the  pilot  could  even 
respond  to  the  failure. 

Fault  warning  and  mode  progress  display  design.  Including  the  pre-experi- 
mental  subject  pilots,  l6  of  the  22  pilots  did  not  like  the  present  system. 

The  overwhelming  response  was  that  the  display  was  too  complex,  confusing, 
had  too  many  lights  and  the  flags  were  too  small.  Some  very  constructive 
suggestions  emeiged,  the  most  significant  being  the  suggestion  to  separate 
the  mode  progress  from  fault  warning  information.  (See  pages  9,  13,  l4. ) 
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Mode  progress  information.  Twelve  pilots  responded  to  a  request  for  sug¬ 
gestions  to  improve  mode  progress  display.  In  general  terms,  they  said 
de-emphasize  or  eliminate  the  display.  It  was  very  clearly  pointed  out  that 
the  mode  progress  function  caused  lights  to  illuminate  to  indicate  normal 
sequencing  of  the  autopilot  in  the  same  location  that  the  FLAG,  CAUTION  and 
DEVIATION  lights  were  located.  Thus,  lights  illuminating  in  the  area  above 
the  ADI  meant  either  normal  or  emergency  events. 

A  composite  of  the  best  responses  indicated  that  the  mode  progress  dis¬ 
play  should  be  (l)  moved  to  the  left  side  of  the  Captain's  display  panel,  (2) 
be  vertical  in  orientation,  and  (3)  that  the  modes  should  be  legible  prior 
to  illumination.  One  pilot  suggested  an  amber  armed  condition  and  a  green 
engagement  condition.  The  reason  for  legibility  without  illuminati^-i  is  so 
that  the  pilots  can  learn  where  the  lights  are  and  know  what  is  supposed  to 
happen  next.  When  the  next  light  turns  on,  they  would  not  have  to  stop  what 
they  are  doing  at  precisely  the  moment  the  light  turns  on  in  order  to  know 
what  the  light  is  indicating. 

Display  flags.  As  previously  indicated,  the  ADI,  HSI  and  Radar  Altimeter 
flags  were  too  small.  Pilots  universally  agreed  that  these  flags  should  be 
large,  international  orange  in  color,  shape  coded,  and  should  say  what  ele¬ 
ment  they  represent  (Fig.  M-3,  p  10). 

Annunciator  design.  The  autopilot  axis  annunciator  was  well  received. 

The  system  annunciator  panel  that  reflected  the  display  system  elements  and 
flight  director  was  somewhat  of  a  problem  in  its  present  design.  The  main 
problem  seemed  to  be  that  the  elements  were  not  coded  such  that  the  element 
that  had  failed  was  obvious  without  reading  the  lights.  In  addition,  the 
systems  affected  by  the  failure  would  also  be  indicated.  For  example,  a 
vertical  gyro  failure  would  disable  the  flight  director  in  pitch.  Both 
lights  would  illuminate  as  is  appropriate  since  both  systems  were  affected. 

It  was  suggested,  however,  that  the  element  failed  be  brighter  than  the 
systems  affected  for  more  rapid  detection  of  the  failed  element  (Fig.  M-8,pl4). 

Another  suggestion  was  that  annunciator  lights  of  relatively  narrow 
design  be  placed  in  a  row  on  both  sides  of  the  ADI-HSI  group,  and  right  up 
against  the  instruments.  Lights  in  the  left  column  would  be  for  the  Captain's 
instruments  and  those  cn  the  right  column  would  be  for  the  First  Officer's 
instruments.  The  annunciator  design  might  even  be  incorporated  into  the 
ADI-HSI  hardware.  This  idea  was  interesting  because  it  suggested  that  the 
fault  warning  annunciator  function  should  be  located  at  the  ADI-HSI.  If,  as 
earlier  discussed,  good  flags  can  be  shewn  to  elicit  pilot  detection  capa¬ 
bilities  that  are  equivalent  to  lights,  all  of  the  display  annunciators  can 
be  represented  by  more  modern  flight  director  systems. 

A  fourth  suggestion  was  to  make  part  of  the  instrument  case  out  of 
electro-luminescent  material  that  would  light  red  if  the  function  failed. 

Thus,  a  radar  altimeter  failure  would  be  indicated  by  the  radar  instrument 
itself  glowing  or  flashing  red  or  orange.  The  caution  light  would  be  retain¬ 
ed  to  reflect  a  problem  anywhere. 
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International orange  was  the  color  most  suggested  for  annunciators.  One- 
half  of  the  pilots  were  in  favor  of  flashing  annunciators;  cne-half  were 
opposed. 

No  subject  suggested  auditory  voice  warning  such  as  the  system  used  on  the 
B-58,  which  uses  a  voice  message  to  indicate  the  problem.  Several  subjects 
suggested  auditory  warnings  in  addition  to  the  caution  light  as  an  alerting 
device . 

Fault-warning  system  design  and  learning.  The  single  most  important 
design  characteristic  of  fault-warning  system  displays  is  that  the  display 
MUST  be  intuitively  obvious.  No  matter  what  question  was  asked,  almost  all 
pilot  responses  contained  elements  of  this  requirement.  The  reason  is  that 
fault-warning  systems  do  not  normally  indicate  anything.  Only  when  a  fail¬ 
ure  occurs  is  the  system  exercized.  How  often,  for  example,  does  a  commercial 
Captain  see  a  warning  flag  on  a  modern  flight  director?  How  often  does  he 
see  all  flags?  Unless  the  fault -monitor  system  false-alarms  frequently,  how 
often  will  the  Captain  see  the  elements  of  the  fault-warning  display?  If 
the  avionic  sub-systems  meet  the  reliability  standards  that  are  being  speci¬ 
fied,  the  flight  crew  may  see  the  entire  fault-warning  system  exercised  only 
during  periodic  simulator  training.  Such  training  must  be  good  enough 
that  the  flight  crew  will  retain  their  skill  in  interpe reting  failures  and 
taking  corrective  action.  For  Category  III  operations,  there  is  no  time 
margin  for  "reviewing  ground  school, "in  the  words  of  one  subject  pilot. 

This  being  the  case,  the  fault-warning  display  system  must  be  so  intuitively 
obvious  that  having  once  learned  the  system  operation,  the  interpretation 
of  the  display  will  be  permanently  retained.  The  design  of  the  system  and 
its  method  of  annunciating  problems  to  the  flight  crew  can  go  a  long  way 
toward  establishing  an  intuitively  obvious  display.  Design  features  like 
covering  failed  functions,  biasing  failed  information  from  view,  or 
illuminating  failed  instruments  are  certainly  more  intuitively  obvious  than 
an  annunciator  panel.  Design  features  like  the  above  along  with  annuncia¬ 
tions  that  suggest  the  proper  course  of  action  (or  alternative  courses ) 
indicate  the  design  direction  for  fault -warning  displays. 
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CONCLUSIONS 


The  major  conclusions  drawn  from  this  study  are: 

1.  Softover  control  failures  were  not  a  pilot  detection  problem. 

2.  Passive  control  failures  require  a  full  monitor  system. 

3.  Autopilot  flare  failures  without  monitor  system  annunciation  were 
frequently  missed. 

4.  Pilots  took  longer  to  detect  display  flags  outside  the  ADI  than 
flags  inside  the  ADI. 

5.  Second  failures  were  frequently  undetected;  the  pilot  was  manual  in 
one  axis  during  second  failures. 

6.  Generally,  the  incorrect  responses,  incorrect  decisions  and  per¬ 
formance  data  suggest  that  additional  monitor  system  improvement  is 
necessary. 

a.  Light  regrouping,  audio  warning  and  larger  flags  should  be  tested. 

b.  Coordinated  crew  procedures  should  also  help. 

7.  Go-around  performance  was  good  in  the  vertical  plane,  but  lateral 
problems  were  evident. 

8.  Split-axes  touchdowns  in  this  study  were  not  successful. 

9.  Rollout  on  raw  information  was  unsatisfactory;  simulation  problems 
emerged. 


10.  These  conclusions  are  restricted  to  the  context  of  the  present  study. 
There  was  no  crew  member  to  help  the  pilots  find  failures.  On  the  other 
hand,  on  each  and  every  approach,  the  pilots  could  expect  a  failure;  per¬ 
formance  differences  occurred  in  spite  of  the  fact  that  pilots  were  looking 
for  failures  and  their  scan  patterns  were  restricted  to  the  inside  of  the 
cockpit  (they  could  not  look  out). 


107 


REFERENCES 


1.  Duncan,  D,  B.  Multiple  Range  and  Multiple  F.  Tests.  Biometrics. 

1955,  11,  1-42 

2.  Edwards,  A.  L.  Experimental  Design  in  Psychological  Research. 

New  York:  Holt,  1965 

3.  Gainer,  C.  A.,  Welde,  W.  L.  &  Monroe,  R.  Cockpit  Displays  for  All- 

Weather  Landings.  Baltimore:  The  Martin  Company, 

AF33 (616) -7752,  E.  R.  12,  446,  May  1962. 

4.  Gainer,  C.  A.,  Monroe,  R.  D.,  Brown,  J.E.,  Bertone,  C.  M. ,  Nichols,  D.  E., 

Dent,  P.  L.  &  Bamebey,  S.  F.  All-Weather  Landings  Simulation  for 
Category  III  Airborne  Configurations^  Volume  I.  Summary  of 
Studies  on  Flight  Directors  and  Split-Axis  Control,  ifoe  Bunker -Ramo 
Corporation,  Canoga  Park,  California,  64WA-5143,  July  1967. 

5.  Lindquist,  E.  F.  Design  and  Analysis  of  Experiments  in  Psychology  and 

Education.  Boston:  Houghton  Mifflin,  1953. 

6.  Obermayer,  R.  W.  &  Muckier,  F.A.  Performance  Measurement  in  Flight 

Simulation  Studies.  Paper  read  at  the  American  Institute  of 
Aeronautics,  National  Specialists  meeting  on  simulation  for 
aerospace  flight,  Columbus,  Ohio:  August  1963. 

7.  Siegel,  S.  Non-Parameteric  Statistics.  New  York:  McGraw-Hill,  1956. 

8.  Stein,  Kenneth  J.,  Monitor  Aids  C-l4l  All  Weather  Landing.  Aviation 

Week  and  Space  Technology.  McGraw-Hill,  April  1967. 


109 


APPENDIX  A 
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APIENDIX  A 


TABLE  A-l 

Analysis  of  Variance  Source  Table  for  Go-Arounds 


Source 

Degrees  of  Freedom 

Total 

202 

Control 

1 

Failure 

2 

Control  X  Failure 

2 

Error 

197 

TABLE  A-2 

Analysis  of  Variance  Source  Table 

for  Touchdowns 

Source 

Degrees  of  Freedom 

Total 

321 

Control 

2 

Failure 

Control  X  Failure 


Error 


2 

4 

313 


TABLE  A-3 


Analysis 

of  Variance  Soiree  Table  for  Control  Failures 

Source 

Degrees  of  Freedom 

Subject 

17 

Group 

1 

Error 

16 

Control 

1 

Failure 

2 

IC 

1 

IF 

2 

CF 

2 

IG 

1 

CG 

1 

FG 

2 

ICG 

1 

IFC 

2 

CFG 

2 

ICF 

2 

ICFG 

9 

Error 

176 

IS 

l6 

CS 

16 

FS 

32 

ICS 

16 

US 

32 

CFS 

32 

ICFS 

32 

Total 

215 

-2 


TABLE  A -4 

Analysis  of  Variance  Source  Table  for  Display  Failures 


Source 

Degrees  of  Freedom 

Subject 

17 

Group 

1 

Error 

16 

Display 

3 

Control 

2 

Failure 

2 

DC 

6 

DF 

6 

CF 

4 

DG 

3 

CG 

2 

FG 

2 

DCG 

6 

DFG 

6 

CFG 

4 

DCF 

12 

DCFG 

12 

Error 

560 

DS 

48 

CS 

32 

FS 

32 

DCS 

96 

DFS 

96 

CFS 

64 

DCFS 

192 

Total 

647 

A-3 
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TAB1£  A-5 

Analysis  of  Variance  Source  Table  for  Flare  Mode  Failure 


Source 

Degrees  of  Freedom 

Subject 

17 

Group 

1 

Error 

16 

Failure 

2 

FG 

2 

Error 

32 

FS 

32 

Total 

53 

A  -k 


